[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260616 19:56:49.601338 6782 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.159.190:36419
I20260616 19:56:49.602969 6782 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260616 19:56:49.604084 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:56:49.621845 6789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:56:49.621878 6788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:56:49.622160 6791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:56:49.623191 6782 server_base.cc:1061] running on GCE node
I20260616 19:56:50.816903 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:56:50.817207 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:56:50.817412 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639810817385 us; error 0 us; skew 500 ppm
I20260616 19:56:50.818248 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:56:50.824391 6782 webserver.cc:533] Webserver started at http://127.6.159.190:40745/ using document root <none> and password file <none>
I20260616 19:56:50.825297 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:56:50.825546 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:56:50.826037 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:56:50.832059 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/master-0-root/instance:
uuid: "1784bd25ee1c42c6891d1a6d818bb721"
format_stamp: "Formatted at 2026-06-16 19:56:50 on dist-test-slave-1c0m"
I20260616 19:56:50.840095 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260616 19:56:50.845366 6797 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:50.846499 6782 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260616 19:56:50.846863 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/master-0-root
uuid: "1784bd25ee1c42c6891d1a6d818bb721"
format_stamp: "Formatted at 2026-06-16 19:56:50 on dist-test-slave-1c0m"
I20260616 19:56:50.847231 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:56:50.906535 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:56:50.908217 6782 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260616 19:56:50.908727 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:56:50.984017 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.190:36419
I20260616 19:56:50.984098 6848 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.190:36419 every 8 connection(s)
I20260616 19:56:50.992117 6849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:51.015196 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721: Bootstrap starting.
I20260616 19:56:51.021308 6849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:51.023296 6849 log.cc:826] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721: Log is configured to *not* fsync() on all Append() calls
I20260616 19:56:51.028589 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721: No bootstrap required, opened a new log
I20260616 19:56:51.052515 6849 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1784bd25ee1c42c6891d1a6d818bb721" member_type: VOTER }
I20260616 19:56:51.053188 6849 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:51.053503 6849 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1784bd25ee1c42c6891d1a6d818bb721, State: Initialized, Role: FOLLOWER
I20260616 19:56:51.054600 6849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1784bd25ee1c42c6891d1a6d818bb721" member_type: VOTER }
I20260616 19:56:51.055222 6849 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260616 19:56:51.055569 6849 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260616 19:56:51.055999 6849 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:51.060846 6849 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1784bd25ee1c42c6891d1a6d818bb721" member_type: VOTER }
I20260616 19:56:51.061681 6849 leader_election.cc:304] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1784bd25ee1c42c6891d1a6d818bb721; no voters:
I20260616 19:56:51.063666 6849 leader_election.cc:290] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260616 19:56:51.064005 6852 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:51.067203 6852 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 1 LEADER]: Becoming Leader. State: Replica: 1784bd25ee1c42c6891d1a6d818bb721, State: Running, Role: LEADER
I20260616 19:56:51.068256 6852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1784bd25ee1c42c6891d1a6d818bb721" member_type: VOTER }
I20260616 19:56:51.069641 6849 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [sys.catalog]: configured and running, proceeding with master startup.
I20260616 19:56:51.078783 6853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1784bd25ee1c42c6891d1a6d818bb721. Latest consensus state: current_term: 1 leader_uuid: "1784bd25ee1c42c6891d1a6d818bb721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1784bd25ee1c42c6891d1a6d818bb721" member_type: VOTER } }
I20260616 19:56:51.078959 6854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1784bd25ee1c42c6891d1a6d818bb721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1784bd25ee1c42c6891d1a6d818bb721" member_type: VOTER } }
I20260616 19:56:51.079595 6854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [sys.catalog]: This master's current role is: LEADER
I20260616 19:56:51.080299 6853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [sys.catalog]: This master's current role is: LEADER
I20260616 19:56:51.084666 6859 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
I20260616 19:56:51.097710 6859 catalog_manager.cc:1519] Initializing Kudu cluster ID...
I20260616 19:56:51.103472 6782 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260616 19:56:51.118472 6859 catalog_manager.cc:1382] Generated new cluster ID: 5a88bbe99c3b458592a483e91339f10e
I20260616 19:56:51.118791 6859 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
I20260616 19:56:51.166306 6859 catalog_manager.cc:1405] Generated new certificate authority record
I20260616 19:56:51.168368 6859 catalog_manager.cc:1539] Loading token signing keys...
I20260616 19:56:51.184870 6859 catalog_manager.cc:6069] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721: Generated new TSK 0
I20260616 19:56:51.185958 6859 catalog_manager.cc:1549] Initializing in-progress tserver states...
I20260616 19:56:51.235033 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:56:51.244680 6871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:56:51.246073 6872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:56:51.249748 6874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:56:51.250553 6782 server_base.cc:1061] running on GCE node
I20260616 19:56:51.251268 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:56:51.251469 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:56:51.251643 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639811251614 us; error 0 us; skew 500 ppm
I20260616 19:56:51.252303 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:56:51.256696 6782 webserver.cc:533] Webserver started at http://127.6.159.129:38717/ using document root <none> and password file <none>
I20260616 19:56:51.257256 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:56:51.257465 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:56:51.257746 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:56:51.258885 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-0-root/instance:
uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
format_stamp: "Formatted at 2026-06-16 19:56:51 on dist-test-slave-1c0m"
I20260616 19:56:51.264990 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260616 19:56:51.269904 6879 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.270776 6782 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260616 19:56:51.271140 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-0-root
uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
format_stamp: "Formatted at 2026-06-16 19:56:51 on dist-test-slave-1c0m"
I20260616 19:56:51.271574 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:56:51.292680 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:56:51.294479 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:56:51.297264 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:56:51.302948 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:56:51.303268 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.303582 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:56:51.303800 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.401621 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.129:45747
I20260616 19:56:51.401711 6941 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.129:45747 every 8 connection(s)
I20260616 19:56:51.409142 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:56:51.421244 6946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:56:51.424096 6947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:56:51.431102 6782 server_base.cc:1061] running on GCE node
W20260616 19:56:51.432293 6949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:56:51.433529 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:56:51.433882 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:56:51.434123 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639811434107 us; error 0 us; skew 500 ppm
I20260616 19:56:51.434888 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:56:51.437856 6782 webserver.cc:533] Webserver started at http://127.6.159.130:42387/ using document root <none> and password file <none>
I20260616 19:56:51.438531 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:56:51.438778 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:56:51.439095 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:56:51.440536 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-1-root/instance:
uuid: "40a2a770d461463e9a88835aab4e6c73"
format_stamp: "Formatted at 2026-06-16 19:56:51 on dist-test-slave-1c0m"
I20260616 19:56:51.442566 6942 heartbeater.cc:344] Connected to a master server at 127.6.159.190:36419
I20260616 19:56:51.442991 6942 heartbeater.cc:461] Registering TS with master...
I20260616 19:56:51.443969 6942 heartbeater.cc:507] Master 127.6.159.190:36419 requested a full tablet report, sending...
I20260616 19:56:51.446949 6814 ts_manager.cc:194] Registered new tserver with Master: 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747)
I20260616 19:56:51.449627 6814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47990
I20260616 19:56:51.449687 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.002s sys 0.007s
I20260616 19:56:51.454780 6954 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.455608 6782 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260616 19:56:51.455950 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-1-root
uuid: "40a2a770d461463e9a88835aab4e6c73"
format_stamp: "Formatted at 2026-06-16 19:56:51 on dist-test-slave-1c0m"
I20260616 19:56:51.456317 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:56:51.489656 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:56:51.491302 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:56:51.494021 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:56:51.497694 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:56:51.497961 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.498243 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:56:51.498453 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.585690 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.130:42141
I20260616 19:56:51.585762 7016 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.130:42141 every 8 connection(s)
I20260616 19:56:51.593528 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:56:51.603150 7020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:56:51.607478 7017 heartbeater.cc:344] Connected to a master server at 127.6.159.190:36419
W20260616 19:56:51.607497 7021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:56:51.608007 7017 heartbeater.cc:461] Registering TS with master...
W20260616 19:56:51.608417 7023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:56:51.608949 7017 heartbeater.cc:507] Master 127.6.159.190:36419 requested a full tablet report, sending...
I20260616 19:56:51.611053 6814 ts_manager.cc:194] Registered new tserver with Master: 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:51.612402 6814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48000
I20260616 19:56:51.618111 6782 server_base.cc:1061] running on GCE node
I20260616 19:56:51.618984 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:56:51.619201 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:56:51.619365 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639811619337 us; error 0 us; skew 500 ppm
I20260616 19:56:51.619947 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:56:51.622545 6782 webserver.cc:533] Webserver started at http://127.6.159.131:45033/ using document root <none> and password file <none>
I20260616 19:56:51.623025 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:56:51.623219 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:56:51.623474 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:56:51.624601 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-2-root/instance:
uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
format_stamp: "Formatted at 2026-06-16 19:56:51 on dist-test-slave-1c0m"
I20260616 19:56:51.629300 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260616 19:56:51.632840 7028 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.633659 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260616 19:56:51.633947 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-2-root
uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
format_stamp: "Formatted at 2026-06-16 19:56:51 on dist-test-slave-1c0m"
I20260616 19:56:51.634271 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1781639809409380-6782-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:56:51.659643 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:56:51.660909 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:56:51.663601 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:56:51.666725 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:56:51.667023 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.667286 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:56:51.667428 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:56:51.763948 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.131:36707
I20260616 19:56:51.764052 7090 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.131:36707 every 8 connection(s)
I20260616 19:56:51.780692 7091 heartbeater.cc:344] Connected to a master server at 127.6.159.190:36419
I20260616 19:56:51.781067 7091 heartbeater.cc:461] Registering TS with master...
I20260616 19:56:51.781850 7091 heartbeater.cc:507] Master 127.6.159.190:36419 requested a full tablet report, sending...
I20260616 19:56:51.783843 6814 ts_manager.cc:194] Registered new tserver with Master: 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:51.785219 6814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48014
I20260616 19:56:51.785475 6782 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017116669s
I20260616 19:56:51.840065 6814 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48016:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260616 19:56:51.843786 6814 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260616 19:56:51.964355 6906 tablet_service.cc:1511] Processing CreateTablet for tablet 7b4ba1bad40148d08f3197e703269a9b (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260616 19:56:51.965407 6905 tablet_service.cc:1511] Processing CreateTablet for tablet 97b4eb44988f4d26beff2c9420faec08 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260616 19:56:51.967686 6904 tablet_service.cc:1511] Processing CreateTablet for tablet 77a7d4ad6e3d4d92a83f12fd31ece99d (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260616 19:56:51.968514 6901 tablet_service.cc:1511] Processing CreateTablet for tablet 1393cafa166044e18db7522c4ee84a83 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260616 19:56:51.970507 6903 tablet_service.cc:1511] Processing CreateTablet for tablet 83fd0e6f6b8e4a34b9f92ffa3f38f63b (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260616 19:56:51.971073 6982 tablet_service.cc:1511] Processing CreateTablet for tablet 3c471229572c4a578e61219cdae70b45 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260616 19:56:51.972534 6906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b4ba1bad40148d08f3197e703269a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:51.977720 6907 tablet_service.cc:1511] Processing CreateTablet for tablet 3c471229572c4a578e61219cdae70b45 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260616 19:56:51.972265 6979 tablet_service.cc:1511] Processing CreateTablet for tablet 77a7d4ad6e3d4d92a83f12fd31ece99d (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260616 19:56:51.985560 6980 tablet_service.cc:1511] Processing CreateTablet for tablet 97b4eb44988f4d26beff2c9420faec08 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260616 19:56:51.986831 6903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83fd0e6f6b8e4a34b9f92ffa3f38f63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:51.978731 6976 tablet_service.cc:1511] Processing CreateTablet for tablet 1393cafa166044e18db7522c4ee84a83 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260616 19:56:51.992342 6902 tablet_service.cc:1511] Processing CreateTablet for tablet 05d07be714a74ac680a9502b788081c1 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260616 19:56:51.990818 6982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c471229572c4a578e61219cdae70b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:51.991688 6900 tablet_service.cc:1511] Processing CreateTablet for tablet eb28b92f7f894493aa2479df8d5f14c9 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260616 19:56:51.998929 6979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a7d4ad6e3d4d92a83f12fd31ece99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:51.999696 6980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97b4eb44988f4d26beff2c9420faec08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:51.998030 6901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1393cafa166044e18db7522c4ee84a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:51.986908 6981 tablet_service.cc:1511] Processing CreateTablet for tablet 7b4ba1bad40148d08f3197e703269a9b (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260616 19:56:52.004144 6904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a7d4ad6e3d4d92a83f12fd31ece99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.005554 6977 tablet_service.cc:1511] Processing CreateTablet for tablet 05d07be714a74ac680a9502b788081c1 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260616 19:56:52.008054 6978 tablet_service.cc:1511] Processing CreateTablet for tablet 83fd0e6f6b8e4a34b9f92ffa3f38f63b (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260616 19:56:52.016955 6976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1393cafa166044e18db7522c4ee84a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.018210 6902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05d07be714a74ac680a9502b788081c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.020943 6975 tablet_service.cc:1511] Processing CreateTablet for tablet eb28b92f7f894493aa2479df8d5f14c9 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260616 19:56:52.020300 6899 tablet_service.cc:1511] Processing CreateTablet for tablet df344a41f1f445ab825428978e0e8c87 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260616 19:56:52.026023 6981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b4ba1bad40148d08f3197e703269a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.021060 7056 tablet_service.cc:1511] Processing CreateTablet for tablet 3c471229572c4a578e61219cdae70b45 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260616 19:56:52.033811 7055 tablet_service.cc:1511] Processing CreateTablet for tablet 7b4ba1bad40148d08f3197e703269a9b (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260616 19:56:52.035810 7056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c471229572c4a578e61219cdae70b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.037132 6975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb28b92f7f894493aa2479df8d5f14c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.037469 6907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c471229572c4a578e61219cdae70b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.041653 7055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b4ba1bad40148d08f3197e703269a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.031774 7053 tablet_service.cc:1511] Processing CreateTablet for tablet 77a7d4ad6e3d4d92a83f12fd31ece99d (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260616 19:56:52.047492 6977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05d07be714a74ac680a9502b788081c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.055953 7053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a7d4ad6e3d4d92a83f12fd31ece99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.049016 7054 tablet_service.cc:1511] Processing CreateTablet for tablet 97b4eb44988f4d26beff2c9420faec08 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260616 19:56:52.057808 6978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83fd0e6f6b8e4a34b9f92ffa3f38f63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.056360 6899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df344a41f1f445ab825428978e0e8c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.054908 6974 tablet_service.cc:1511] Processing CreateTablet for tablet df344a41f1f445ab825428978e0e8c87 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260616 19:56:52.067317 6905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97b4eb44988f4d26beff2c9420faec08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.066406 7050 tablet_service.cc:1511] Processing CreateTablet for tablet 1393cafa166044e18db7522c4ee84a83 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260616 19:56:52.059432 7049 tablet_service.cc:1511] Processing CreateTablet for tablet eb28b92f7f894493aa2479df8d5f14c9 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260616 19:56:52.073158 6974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df344a41f1f445ab825428978e0e8c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.075559 7048 tablet_service.cc:1511] Processing CreateTablet for tablet df344a41f1f445ab825428978e0e8c87 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260616 19:56:52.064390 7052 tablet_service.cc:1511] Processing CreateTablet for tablet 83fd0e6f6b8e4a34b9f92ffa3f38f63b (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260616 19:56:52.063380 7051 tablet_service.cc:1511] Processing CreateTablet for tablet 05d07be714a74ac680a9502b788081c1 (DEFAULT_TABLE table=test-workload [id=16dc5e6693cb47b68642d80e8357981a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260616 19:56:52.092769 6900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb28b92f7f894493aa2479df8d5f14c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260616 19:56:52.095016 6862 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:56:52.096220 7054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97b4eb44988f4d26beff2c9420faec08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.127429 7048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df344a41f1f445ab825428978e0e8c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.131815 7050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1393cafa166044e18db7522c4ee84a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.143106 7052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83fd0e6f6b8e4a34b9f92ffa3f38f63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.149922 7049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb28b92f7f894493aa2479df8d5f14c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.195019 7051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05d07be714a74ac680a9502b788081c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:56:52.278990 7109 tablet_bootstrap.cc:492] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:52.287523 7110 tablet_bootstrap.cc:492] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:52.328715 7111 tablet_bootstrap.cc:492] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
W20260616 19:56:52.392088 7098 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.6.159.190:36419, user_credentials={real_user=slave}} blocked reactor thread for 69359.9us
I20260616 19:56:52.398975 7110 tablet_bootstrap.cc:654] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.409703 7110 tablet_bootstrap.cc:492] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:52.410000 7109 tablet_bootstrap.cc:654] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.409592 7111 tablet_bootstrap.cc:654] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.410252 7110 ts_tablet_manager.cc:1403] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.124s user 0.011s sys 0.004s
I20260616 19:56:52.414203 7110 raft_consensus.cc:359] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.415298 7110 raft_consensus.cc:385] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.415683 7110 raft_consensus.cc:740] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.416795 7110 consensus_queue.cc:260] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.426324 7110 ts_tablet_manager.cc:1434] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260616 19:56:52.425977 7091 heartbeater.cc:499] Master 127.6.159.190:36419 was elected leader, sending a full tablet report...
I20260616 19:56:52.427237 7110 tablet_bootstrap.cc:492] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:52.427484 7109 tablet_bootstrap.cc:492] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:52.427919 7109 ts_tablet_manager.cc:1403] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.201s user 0.019s sys 0.013s
I20260616 19:56:52.430948 7109 raft_consensus.cc:359] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.431705 7109 raft_consensus.cc:385] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.432001 7109 raft_consensus.cc:740] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.433652 7109 consensus_queue.cc:260] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.434417 7110 tablet_bootstrap.cc:654] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.441897 7017 heartbeater.cc:499] Master 127.6.159.190:36419 was elected leader, sending a full tablet report...
I20260616 19:56:52.442907 7109 ts_tablet_manager.cc:1434] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260616 19:56:52.443799 7109 tablet_bootstrap.cc:492] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:52.447376 7111 tablet_bootstrap.cc:492] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:52.447880 7111 ts_tablet_manager.cc:1403] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.160s user 0.015s sys 0.013s
I20260616 19:56:52.451481 7109 tablet_bootstrap.cc:654] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.451543 7111 raft_consensus.cc:359] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.452607 6942 heartbeater.cc:499] Master 127.6.159.190:36419 was elected leader, sending a full tablet report...
I20260616 19:56:52.452836 7111 raft_consensus.cc:385] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.453269 7111 raft_consensus.cc:740] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.454198 7111 consensus_queue.cc:260] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.467725 7111 ts_tablet_manager.cc:1434] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.019s user 0.002s sys 0.008s
I20260616 19:56:52.469081 7111 tablet_bootstrap.cc:492] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:52.477802 7110 tablet_bootstrap.cc:492] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:52.478304 7110 ts_tablet_manager.cc:1403] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.051s user 0.026s sys 0.013s
I20260616 19:56:52.478410 7109 tablet_bootstrap.cc:492] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:52.478904 7109 ts_tablet_manager.cc:1403] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.035s user 0.017s sys 0.009s
I20260616 19:56:52.481613 7110 raft_consensus.cc:359] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.482347 7110 raft_consensus.cc:385] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.482774 7110 raft_consensus.cc:740] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.482363 7109 raft_consensus.cc:359] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.483335 7109 raft_consensus.cc:385] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.483664 7109 raft_consensus.cc:740] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.483517 7110 consensus_queue.cc:260] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.484543 7109 consensus_queue.cc:260] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.485683 7110 ts_tablet_manager.cc:1434] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260616 19:56:52.486754 7110 tablet_bootstrap.cc:492] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:52.486905 7109 ts_tablet_manager.cc:1434] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260616 19:56:52.487882 7109 tablet_bootstrap.cc:492] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:52.493871 7110 tablet_bootstrap.cc:654] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.494601 7109 tablet_bootstrap.cc:654] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
W20260616 19:56:52.522194 7092 tablet.cc:2406] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260616 19:56:52.529562 7111 tablet_bootstrap.cc:654] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.549820 7116 raft_consensus.cc:493] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.560639 7116 raft_consensus.cc:515] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.565451 7116 leader_election.cc:290] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.573261 7109 tablet_bootstrap.cc:492] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:52.573757 7109 ts_tablet_manager.cc:1403] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.086s user 0.040s sys 0.032s
I20260616 19:56:52.583041 7109 raft_consensus.cc:359] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.583825 7109 raft_consensus.cc:385] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.584136 7109 raft_consensus.cc:740] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.585094 7111 tablet_bootstrap.cc:492] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:52.585369 7109 consensus_queue.cc:260] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.585743 7111 ts_tablet_manager.cc:1403] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.117s user 0.010s sys 0.021s
I20260616 19:56:52.588079 7109 ts_tablet_manager.cc:1434] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.014s user 0.006s sys 0.002s
I20260616 19:56:52.589113 7109 tablet_bootstrap.cc:492] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:52.595777 7109 tablet_bootstrap.cc:654] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.597185 7110 tablet_bootstrap.cc:492] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:52.597672 7110 ts_tablet_manager.cc:1403] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.111s user 0.030s sys 0.049s
I20260616 19:56:52.608278 7109 tablet_bootstrap.cc:492] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:52.608769 7109 ts_tablet_manager.cc:1403] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.005s
I20260616 19:56:52.609261 7111 raft_consensus.cc:359] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.610031 7111 raft_consensus.cc:385] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.610353 7111 raft_consensus.cc:740] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.611084 7111 consensus_queue.cc:260] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.612066 7109 raft_consensus.cc:359] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.612805 7109 raft_consensus.cc:385] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.613101 7109 raft_consensus.cc:740] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.613528 7111 ts_tablet_manager.cc:1434] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.010s user 0.001s sys 0.005s
I20260616 19:56:52.613801 7109 consensus_queue.cc:260] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.614434 7111 tablet_bootstrap.cc:492] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:52.616676 7109 ts_tablet_manager.cc:1434] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.008s user 0.001s sys 0.006s
I20260616 19:56:52.617568 7109 tablet_bootstrap.cc:492] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:52.617445 7110 raft_consensus.cc:359] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.618160 7110 raft_consensus.cc:385] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.618458 7110 raft_consensus.cc:740] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.624243 7109 tablet_bootstrap.cc:654] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.619573 7110 consensus_queue.cc:260] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.630847 7110 ts_tablet_manager.cc:1434] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20260616 19:56:52.631870 7110 tablet_bootstrap.cc:492] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:52.636843 7111 tablet_bootstrap.cc:654] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.640843 7110 tablet_bootstrap.cc:654] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.646544 7066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:52.647810 7066 raft_consensus.cc:2468] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 0.
I20260616 19:56:52.649785 7109 tablet_bootstrap.cc:492] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:52.650410 6956 leader_election.cc:304] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:56:52.651093 7109 ts_tablet_manager.cc:1403] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.012s
I20260616 19:56:52.651463 7116 raft_consensus.cc:2804] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:52.651840 7116 raft_consensus.cc:493] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:52.652621 7116 raft_consensus.cc:3060] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.659612 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:52.660398 7116 raft_consensus.cc:515] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
W20260616 19:56:52.662953 6957 leader_election.cc:343] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.663451 7066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:52.663993 7066 raft_consensus.cc:3060] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.664454 7116 leader_election.cc:290] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.665087 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
W20260616 19:56:52.666710 6957 leader_election.cc:343] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.668315 7109 raft_consensus.cc:359] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.669080 7109 raft_consensus.cc:385] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.669436 7109 raft_consensus.cc:740] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.672516 7109 consensus_queue.cc:260] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.677089 7066 raft_consensus.cc:2468] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 1.
I20260616 19:56:52.677626 7109 ts_tablet_manager.cc:1434] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.026s user 0.003s sys 0.005s
I20260616 19:56:52.678640 7109 tablet_bootstrap.cc:492] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:52.678788 6956 leader_election.cc:304] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters: 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:52.681021 7110 tablet_bootstrap.cc:492] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:52.681670 7116 raft_consensus.cc:2804] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:52.681792 7110 ts_tablet_manager.cc:1403] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.050s user 0.011s sys 0.007s
I20260616 19:56:52.682155 7116 raft_consensus.cc:697] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:52.683058 7116 consensus_queue.cc:237] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.686946 7118 raft_consensus.cc:493] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.687073 7110 raft_consensus.cc:359] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.687755 7110 raft_consensus.cc:385] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.687484 7118 raft_consensus.cc:515] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.688024 7110 raft_consensus.cc:740] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.688752 7110 consensus_queue.cc:260] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.690310 7118 leader_election.cc:290] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.692541 7110 ts_tablet_manager.cc:1434] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20260616 19:56:52.703629 7111 tablet_bootstrap.cc:492] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:52.704137 7111 ts_tablet_manager.cc:1403] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.090s user 0.014s sys 0.006s
I20260616 19:56:52.711277 7111 raft_consensus.cc:359] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.708415 7110 tablet_bootstrap.cc:492] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:52.712051 7111 raft_consensus.cc:385] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.717662 7111 raft_consensus.cc:740] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.722674 7111 consensus_queue.cc:260] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.724875 7111 ts_tablet_manager.cc:1434] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.020s user 0.005s sys 0.002s
I20260616 19:56:52.725849 7111 tablet_bootstrap.cc:492] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:52.729179 7109 tablet_bootstrap.cc:654] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.737107 7116 raft_consensus.cc:493] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.737922 7116 raft_consensus.cc:515] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.740639 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:52.740897 7066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:52.739919 7116 leader_election.cc:290] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.743491 7131 raft_consensus.cc:493] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.743942 7118 raft_consensus.cc:493] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.742498 6917 raft_consensus.cc:2468] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 0.
I20260616 19:56:52.744351 7131 raft_consensus.cc:515] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.744590 7118 raft_consensus.cc:515] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.748339 6957 leader_election.cc:304] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:52.749344 7118 leader_election.cc:290] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
W20260616 19:56:52.749339 6956 leader_election.cc:343] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.749739 7116 raft_consensus.cc:2804] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:52.750135 7116 raft_consensus.cc:493] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:52.750460 7116 raft_consensus.cc:3060] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.754366 7115 raft_consensus.cc:493] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.754837 7115 raft_consensus.cc:515] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.758102 7066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:52.760401 7115 leader_election.cc:290] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:52.754366 7131 leader_election.cc:290] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.773811 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
W20260616 19:56:52.781801 6957 leader_election.cc:343] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.792312 7066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:52.795296 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
W20260616 19:56:52.797544 6881 leader_election.cc:343] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.793823 7115 raft_consensus.cc:493] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.750998 6813 catalog_manager.cc:5696] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:52.815837 7111 tablet_bootstrap.cc:654] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.815635 7115 raft_consensus.cc:515] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.822713 7065 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:52.824609 7115 leader_election.cc:290] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:52.825009 7109 tablet_bootstrap.cc:492] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:52.827015 7109 ts_tablet_manager.cc:1403] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.149s user 0.030s sys 0.015s
I20260616 19:56:52.829716 7110 tablet_bootstrap.cc:654] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
W20260616 19:56:52.829689 6956 leader_election.cc:343] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.831558 6956 leader_election.cc:304] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73; no voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:52.831843 6881 leader_election.cc:304] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:52.833053 7118 raft_consensus.cc:2804] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:52.833472 7118 raft_consensus.cc:493] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:52.833110 7131 raft_consensus.cc:2749] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:56:52.833789 7118 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.818260 7116 raft_consensus.cc:515] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.838032 7109 raft_consensus.cc:359] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.839450 7109 raft_consensus.cc:385] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.841799 7109 raft_consensus.cc:740] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.856254 7118 raft_consensus.cc:515] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.860016 7109 consensus_queue.cc:260] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.861840 7116 leader_election.cc:290] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.864852 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:52.867319 6991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:52.870570 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:52.866549 7118 leader_election.cc:290] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Requested vote from peers 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:52.870512 6990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:52.873063 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:52.874357 6917 raft_consensus.cc:3060] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.869959 6992 raft_consensus.cc:2393] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4cac913e39174d1c9fd124a69d6e4f6c in current term 1: Already voted for candidate 40a2a770d461463e9a88835aab4e6c73 in this term.
W20260616 19:56:52.876434 6883 leader_election.cc:343] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Illegal state: must be running to vote when last-logged opid is not known
W20260616 19:56:52.877229 6883 leader_election.cc:343] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.877151 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
W20260616 19:56:52.879504 6956 leader_election.cc:343] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.878756 7065 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.880914 6883 leader_election.cc:304] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cac913e39174d1c9fd124a69d6e4f6c; no voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73
I20260616 19:56:52.884752 6916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:52.886212 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:52.887598 7109 ts_tablet_manager.cc:1434] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.060s user 0.009s sys 0.000s
I20260616 19:56:52.889465 6915 raft_consensus.cc:2393] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1c96bff2cfee441cad5e5acecd2bca1d in current term 1: Already voted for candidate 4cac913e39174d1c9fd124a69d6e4f6c in this term.
I20260616 19:56:52.889593 6991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
W20260616 19:56:52.891907 7032 leader_election.cc:343] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.892654 7109 tablet_bootstrap.cc:492] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:52.891827 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:52.896893 7065 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
W20260616 19:56:52.897603 7031 leader_election.cc:343] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.898792 6881 leader_election.cc:304] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters: 40a2a770d461463e9a88835aab4e6c73
W20260616 19:56:52.898440 7032 leader_election.cc:343] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.898845 7031 leader_election.cc:304] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d; no voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:52.903313 7115 raft_consensus.cc:2749] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:56:52.904759 7032 leader_election.cc:304] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d; no voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:52.905015 7141 raft_consensus.cc:493] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.907620 7115 raft_consensus.cc:2749] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:56:52.907831 7111 tablet_bootstrap.cc:492] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:52.907034 6917 raft_consensus.cc:2468] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 1.
I20260616 19:56:52.910766 7118 raft_consensus.cc:2749] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:56:52.907805 7141 raft_consensus.cc:515] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.911218 7111 ts_tablet_manager.cc:1403] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.186s user 0.010s sys 0.022s
I20260616 19:56:52.914041 6957 leader_election.cc:304] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters: 1c96bff2cfee441cad5e5acecd2bca1d
I20260616 19:56:52.915539 7118 raft_consensus.cc:2804] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:52.916610 7116 raft_consensus.cc:2804] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:52.917261 7116 raft_consensus.cc:697] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:52.920465 7118 raft_consensus.cc:697] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:52.922072 7118 consensus_queue.cc:237] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:52.922468 7116 consensus_queue.cc:237] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.928647 7109 tablet_bootstrap.cc:654] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.936990 7110 tablet_bootstrap.cc:492] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:52.937640 7110 ts_tablet_manager.cc:1403] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.239s user 0.019s sys 0.013s
I20260616 19:56:52.937572 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:52.941709 7111 raft_consensus.cc:359] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
W20260616 19:56:52.942353 6883 leader_election.cc:343] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.941955 7110 raft_consensus.cc:359] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.942628 7111 raft_consensus.cc:385] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.942884 7110 raft_consensus.cc:385] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:52.942989 7111 raft_consensus.cc:740] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.943272 7110 raft_consensus.cc:740] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:52.943929 7111 consensus_queue.cc:260] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.944082 7110 consensus_queue.cc:260] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.947090 7111 ts_tablet_manager.cc:1434] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.031s user 0.006s sys 0.001s
I20260616 19:56:52.947423 6814 catalog_manager.cc:5696] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 1 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:52.948607 7111 tablet_bootstrap.cc:492] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:52.950824 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:52.948944 7141 leader_election.cc:290] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.951463 7065 raft_consensus.cc:2468] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:52.952811 6881 leader_election.cc:304] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters: 40a2a770d461463e9a88835aab4e6c73
I20260616 19:56:52.956717 7116 raft_consensus.cc:493] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.957201 7116 raft_consensus.cc:515] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:52.958323 7111 tablet_bootstrap.cc:654] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:52.958489 7141 raft_consensus.cc:2804] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:52.958894 7141 raft_consensus.cc:493] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:52.959237 7141 raft_consensus.cc:3060] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.959791 7116 leader_election.cc:290] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:52.960629 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:52.961386 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
W20260616 19:56:52.962177 6957 leader_election.cc:343] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.964550 7115 raft_consensus.cc:493] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:52.965011 7115 raft_consensus.cc:515] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
W20260616 19:56:52.966341 6956 leader_election.cc:343] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:52.966763 6956 leader_election.cc:304] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73; no voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:52.967514 7116 raft_consensus.cc:2749] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:56:52.970804 7115 leader_election.cc:290] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:52.972453 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:52.973074 6917 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 0.
I20260616 19:56:52.976017 7031 leader_election.cc:304] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:52.976231 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:52.976816 7115 raft_consensus.cc:2804] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:52.977186 7115 raft_consensus.cc:493] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:52.977037 6992 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 0.
I20260616 19:56:52.977540 7115 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:52.978061 7110 ts_tablet_manager.cc:1434] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260616 19:56:52.996735 7141 raft_consensus.cc:515] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.000379 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:53.000633 7141 leader_election.cc:290] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.001572 7109 tablet_bootstrap.cc:492] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:53.002015 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:53.002187 7109 ts_tablet_manager.cc:1403] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.110s user 0.012s sys 0.024s
I20260616 19:56:53.002583 7065 raft_consensus.cc:3060] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:56:53.004017 6883 leader_election.cc:343] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:56:53.004832 7110 tablet_bootstrap.cc:492] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:53.005518 7115 raft_consensus.cc:515] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.015651 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:53.016378 6917 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.018229 7110 tablet_bootstrap.cc:654] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.024861 7115 leader_election.cc:290] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:53.025861 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:53.027194 6992 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.006529 7109 raft_consensus.cc:359] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.028316 7109 raft_consensus.cc:385] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.028673 7109 raft_consensus.cc:740] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.029753 7065 raft_consensus.cc:2468] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
I20260616 19:56:53.029557 6917 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 1.
I20260616 19:56:53.034098 6881 leader_election.cc:304] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters: 40a2a770d461463e9a88835aab4e6c73
I20260616 19:56:53.035327 7141 raft_consensus.cc:2804] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:53.035799 7141 raft_consensus.cc:697] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:53.036705 7141 consensus_queue.cc:237] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.041016 7031 leader_election.cc:304] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:53.040366 7109 consensus_queue.cc:260] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.041733 7110 tablet_bootstrap.cc:492] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:53.042961 7110 ts_tablet_manager.cc:1403] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.007s
I20260616 19:56:53.042358 7115 raft_consensus.cc:2804] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:53.046924 7109 ts_tablet_manager.cc:1434] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.044s user 0.010s sys 0.000s
I20260616 19:56:53.048807 7111 tablet_bootstrap.cc:492] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:53.049283 7111 ts_tablet_manager.cc:1403] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.101s user 0.044s sys 0.034s
I20260616 19:56:53.052569 7109 tablet_bootstrap.cc:492] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:53.057307 7111 raft_consensus.cc:359] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.058653 7111 raft_consensus.cc:385] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.059463 7111 raft_consensus.cc:740] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.063369 7110 raft_consensus.cc:359] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.064069 7110 raft_consensus.cc:385] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.064548 7115 raft_consensus.cc:697] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 LEADER]: Becoming Leader. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:56:53.064563 7110 raft_consensus.cc:740] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.062444 7111 consensus_queue.cc:260] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.067219 7115 consensus_queue.cc:237] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.067080 7110 consensus_queue.cc:260] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.068981 7111 ts_tablet_manager.cc:1434] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.019s user 0.004s sys 0.006s
I20260616 19:56:53.071342 7111 tablet_bootstrap.cc:492] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:53.076180 6992 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 1.
I20260616 19:56:53.079763 7109 tablet_bootstrap.cc:654] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.080467 7110 ts_tablet_manager.cc:1434] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.037s user 0.005s sys 0.005s
I20260616 19:56:53.084833 7110 tablet_bootstrap.cc:492] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:53.084237 7116 consensus_queue.cc:1048] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:56:53.093958 7109 tablet_bootstrap.cc:492] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:53.094424 7109 ts_tablet_manager.cc:1403] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.043s user 0.014s sys 0.000s
I20260616 19:56:53.096881 7111 tablet_bootstrap.cc:654] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
W20260616 19:56:53.101060 6862 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:56:53.104723 6814 catalog_manager.cc:5696] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:53.114867 6813 catalog_manager.cc:5696] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 1 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:53.136754 7109 raft_consensus.cc:359] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:53.138010 7109 raft_consensus.cc:385] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.138996 7109 raft_consensus.cc:740] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.139690 6917 raft_consensus.cc:3060] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.150097 7109 consensus_queue.cc:260] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:53.153801 7109 ts_tablet_manager.cc:1434] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.059s user 0.007s sys 0.004s
I20260616 19:56:53.154775 7109 tablet_bootstrap.cc:492] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73: Bootstrap starting.
I20260616 19:56:53.163472 7110 tablet_bootstrap.cc:654] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.163856 7111 tablet_bootstrap.cc:492] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:53.164443 7111 ts_tablet_manager.cc:1403] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.093s user 0.015s sys 0.037s
I20260616 19:56:53.167681 7111 raft_consensus.cc:359] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.168642 7111 raft_consensus.cc:385] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.169323 7111 raft_consensus.cc:740] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.180606 7111 consensus_queue.cc:260] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.182446 7109 tablet_bootstrap.cc:654] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.183689 7111 ts_tablet_manager.cc:1434] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.018s user 0.007s sys 0.001s
I20260616 19:56:53.183674 6812 catalog_manager.cc:5696] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131). New cstate: current_term: 1 leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: HEALTHY } } }
I20260616 19:56:53.185786 7111 tablet_bootstrap.cc:492] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:53.189009 7110 tablet_bootstrap.cc:492] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:53.189510 7110 ts_tablet_manager.cc:1403] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.105s user 0.004s sys 0.011s
I20260616 19:56:53.192672 7110 raft_consensus.cc:359] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.193419 7110 raft_consensus.cc:385] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.193696 7110 raft_consensus.cc:740] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.194428 7110 consensus_queue.cc:260] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.196712 7110 ts_tablet_manager.cc:1434] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260616 19:56:53.197693 7110 tablet_bootstrap.cc:492] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:53.206736 7161 raft_consensus.cc:493] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:53.207317 7161 raft_consensus.cc:515] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.208791 7111 tablet_bootstrap.cc:654] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.209756 7110 tablet_bootstrap.cc:654] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.211372 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:53.214444 7161 leader_election.cc:290] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.216164 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:53.217088 7065 raft_consensus.cc:2393] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 40a2a770d461463e9a88835aab4e6c73 in current term 1: Already voted for candidate 4cac913e39174d1c9fd124a69d6e4f6c in this term.
I20260616 19:56:53.218231 7109 tablet_bootstrap.cc:492] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73: No bootstrap required, opened a new log
I20260616 19:56:53.218297 6956 leader_election.cc:304] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73; no voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:53.218762 7109 ts_tablet_manager.cc:1403] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73: Time spent bootstrapping tablet: real 0.064s user 0.012s sys 0.002s
I20260616 19:56:53.219275 7161 raft_consensus.cc:3060] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.221382 7109 raft_consensus.cc:359] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.222049 7109 raft_consensus.cc:385] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.222333 7109 raft_consensus.cc:740] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.223004 7109 consensus_queue.cc:260] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.225111 7109 ts_tablet_manager.cc:1434] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260616 19:56:53.227938 7110 tablet_bootstrap.cc:492] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:53.228435 7110 ts_tablet_manager.cc:1403] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.003s
I20260616 19:56:53.227938 7111 tablet_bootstrap.cc:492] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:53.229542 7111 ts_tablet_manager.cc:1403] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.015s
I20260616 19:56:53.231025 7110 raft_consensus.cc:359] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.231717 7110 raft_consensus.cc:385] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.232002 7110 raft_consensus.cc:740] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.232689 7110 consensus_queue.cc:260] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.233067 7111 raft_consensus.cc:359] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.234642 7110 ts_tablet_manager.cc:1434] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260616 19:56:53.233809 7111 raft_consensus.cc:385] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.236879 7110 tablet_bootstrap.cc:492] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d: Bootstrap starting.
I20260616 19:56:53.236928 7111 raft_consensus.cc:740] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.238085 7111 consensus_queue.cc:260] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.240649 7111 ts_tablet_manager.cc:1434] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260616 19:56:53.242045 7111 tablet_bootstrap.cc:492] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:53.244076 7110 tablet_bootstrap.cc:654] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.251559 7111 tablet_bootstrap.cc:654] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.255306 7116 raft_consensus.cc:493] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:53.255678 7110 tablet_bootstrap.cc:492] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d: No bootstrap required, opened a new log
I20260616 19:56:53.256124 7110 ts_tablet_manager.cc:1403] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.007s
I20260616 19:56:53.255822 7116 raft_consensus.cc:515] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.258451 7116 leader_election.cc:290] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.258669 7110 raft_consensus.cc:359] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.259336 7110 raft_consensus.cc:385] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.259987 7110 raft_consensus.cc:740] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.260057 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:53.260428 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:53.260759 6915 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 0.
I20260616 19:56:53.261018 7065 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 0.
I20260616 19:56:53.260702 7110 consensus_queue.cc:260] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.262188 6956 leader_election.cc:304] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:56:53.263041 7110 ts_tablet_manager.cc:1434] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260616 19:56:53.263026 7116 raft_consensus.cc:2804] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:53.263545 7116 raft_consensus.cc:493] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:53.263890 7116 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.272249 7111 tablet_bootstrap.cc:492] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:53.272730 7111 ts_tablet_manager.cc:1403] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.003s
I20260616 19:56:53.275969 7161 raft_consensus.cc:2749] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:56:53.282313 7161 consensus_queue.cc:1048] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:56:53.275534 7111 raft_consensus.cc:359] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.284178 7111 raft_consensus.cc:385] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.284487 7111 raft_consensus.cc:740] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.290517 7111 consensus_queue.cc:260] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.293653 7111 ts_tablet_manager.cc:1434] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260616 19:56:53.294706 7111 tablet_bootstrap.cc:492] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c: Bootstrap starting.
I20260616 19:56:53.301535 7111 tablet_bootstrap.cc:654] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c: Neither blocks nor log segments found. Creating new log.
I20260616 19:56:53.308570 7111 tablet_bootstrap.cc:492] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c: No bootstrap required, opened a new log
I20260616 19:56:53.308972 7111 ts_tablet_manager.cc:1403] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.002s
I20260616 19:56:53.310938 7111 raft_consensus.cc:359] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.311450 7111 raft_consensus.cc:385] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:56:53.311673 7111 raft_consensus.cc:740] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Initialized, Role: FOLLOWER
I20260616 19:56:53.312312 7111 consensus_queue.cc:260] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.314218 7111 ts_tablet_manager.cc:1434] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260616 19:56:53.378823 7116 raft_consensus.cc:515] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.381014 7116 leader_election.cc:290] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.431391 7141 raft_consensus.cc:493] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:53.432009 7141 raft_consensus.cc:515] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.434199 7141 leader_election.cc:290] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.436756 6991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:53.437290 7065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:53.437423 6991 raft_consensus.cc:2468] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:53.437868 7065 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.438529 6883 leader_election.cc:304] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:53.439296 7141 raft_consensus.cc:2804] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:53.439668 7141 raft_consensus.cc:493] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:53.439997 7141 raft_consensus.cc:3060] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.444818 6992 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.810523 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:53.811424 6917 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.821043 7172 raft_consensus.cc:493] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:53.821626 7172 raft_consensus.cc:515] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.830258 7173 raft_consensus.cc:493] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:53.830837 7173 raft_consensus.cc:515] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.832266 7172 leader_election.cc:290] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.827862 7118 consensus_queue.cc:1048] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260616 19:56:53.834405 7173 leader_election.cc:290] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.839396 7174 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260616 19:56:53.840407 6991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:53.841022 6991 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:53.842007 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:53.843103 6992 raft_consensus.cc:2468] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:53.849150 6917 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 1.
I20260616 19:56:53.852486 7176 consensus_queue.cc:1048] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:56:53.853477 7141 raft_consensus.cc:515] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.852923 6957 leader_election.cc:304] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:53.855373 7176 raft_consensus.cc:2804] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:53.855846 7176 raft_consensus.cc:697] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:53.856652 7176 consensus_queue.cc:237] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:53.857551 7141 leader_election.cc:290] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:53.870630 7066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:53.871398 7066 raft_consensus.cc:2468] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:53.872638 6990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:53.873206 6990 raft_consensus.cc:3060] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.885331 7066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:53.894455 7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:53.895181 7064 raft_consensus.cc:2468] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:53.893877 7066 raft_consensus.cc:3060] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.892584 6883 leader_election.cc:304] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:53.898574 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" is_pre_election: true
I20260616 19:56:53.899296 7063 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 0.
I20260616 19:56:53.898660 6883 leader_election.cc:304] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:53.900310 7141 raft_consensus.cc:2804] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260616 19:56:54.061280 6881 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:36707, user_credentials={real_user=slave}} blocked reactor thread for 169405us
I20260616 19:56:54.156546 7064 raft_consensus.cc:3060] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:53.900964 7141 raft_consensus.cc:493] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:54.165149 7141 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:54.164985 7173 consensus_queue.cc:1048] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:56:54.175266 7174 raft_consensus.cc:2804] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:56:54.175802 7174 raft_consensus.cc:493] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:56:54.176268 7174 raft_consensus.cc:3060] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:56:54.203570 6862 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:56:54.161757 6990 raft_consensus.cc:2468] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
I20260616 19:56:54.216791 7181 consensus_queue.cc:1048] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260616 19:56:54.227175 6883 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:42141, user_credentials={real_user=slave}} blocked reactor thread for 328575us
I20260616 19:56:54.233737 7173 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:56:54.234663 6883 leader_election.cc:304] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
W20260616 19:56:54.237064 7065 consensus_meta.cc:287] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d: Time spent flushing consensus metadata: real 0.799s user 0.012s sys 0.015s
I20260616 19:56:54.240646 7065 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 1.
I20260616 19:56:54.240195 7116 consensus_queue.cc:1048] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:56:54.236897 7141 raft_consensus.cc:515] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:54.237885 7118 raft_consensus.cc:2804] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:54.244010 7115 raft_consensus.cc:493] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:56:54.259338 7181 consensus_queue.cc:1048] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260616 19:56:54.260066 7115 raft_consensus.cc:515] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:54.278317 7118 raft_consensus.cc:697] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:54.289978 7141 leader_election.cc:290] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:54.292433 7118 consensus_queue.cc:237] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:54.293370 7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:54.294031 7064 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:54.296691 6990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:54.297288 6990 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:54.327138 7066 raft_consensus.cc:2468] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
I20260616 19:56:54.327390 6990 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
I20260616 19:56:54.329191 6883 leader_election.cc:304] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:54.334327 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" is_pre_election: true
I20260616 19:56:54.327163 7174 raft_consensus.cc:515] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:54.340106 7173 raft_consensus.cc:2804] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:54.345376 7173 raft_consensus.cc:697] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:54.347143 7173 consensus_queue.cc:237] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:54.350931 7174 leader_election.cc:290] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:54.357103 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:54.357244 6990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:54.357959 6990 raft_consensus.cc:3060] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:54.357386 6813 catalog_manager.cc:5696] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:54.357959 7063 raft_consensus.cc:3060] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:56:54.383150 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a2a770d461463e9a88835aab4e6c73" is_pre_election: true
I20260616 19:56:54.377127 7115 leader_election.cc:290] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:54.422055 7032 leader_election.cc:304] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d; no voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:54.422991 7181 raft_consensus.cc:2749] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:56:54.430225 7064 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
I20260616 19:56:54.431224 6990 raft_consensus.cc:2468] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
I20260616 19:56:54.437999 6883 leader_election.cc:304] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:54.439034 7172 raft_consensus.cc:2804] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:56:54.439641 7172 raft_consensus.cc:697] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:54.440646 7172 consensus_queue.cc:237] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:54.456327 7176 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260616 19:56:54.475917 7013 debug-util.cc:398] Leaking SignalData structure 0x7b0800084200 after lost signal to thread 6785
W20260616 19:56:54.476897 7013 debug-util.cc:398] Leaking SignalData structure 0x7b0800083b40 after lost signal to thread 6848
W20260616 19:56:54.477880 7013 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3cc0 after lost signal to thread 6941
W20260616 19:56:54.478734 7013 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3d80 after lost signal to thread 7016
W20260616 19:56:54.479576 7013 debug-util.cc:398] Leaking SignalData structure 0x7b08001a76c0 after lost signal to thread 7090
I20260616 19:56:54.485060 7131 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:56:54.513115 6812 catalog_manager.cc:5696] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 1 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:54.515339 6812 catalog_manager.cc:5696] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 1 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:54.532011 7063 raft_consensus.cc:2468] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 1.
I20260616 19:56:54.551466 6811 catalog_manager.cc:5696] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 1 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:54.782620 7172 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:56:54.804020 7141 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:56:54.816975 7141 consensus_queue.cc:1048] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:56:54.853572 7173 consensus_queue.cc:1048] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:56:54.894675 7141 consensus_queue.cc:1048] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:56:54.910984 7173 consensus_queue.cc:1048] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:56:54.990702 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:54160
I20260616 19:56:54.991328 6992 raft_consensus.cc:606] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Received request to transfer leadership to TS 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:56:55.015228 6917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:35194
I20260616 19:56:55.015897 6917 raft_consensus.cc:606] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:56:55.034642 7064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:40964
I20260616 19:56:55.035292 7064 raft_consensus.cc:606] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:56:55.041271 6917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:35194
I20260616 19:56:55.041900 6917 raft_consensus.cc:606] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:56:55.047670 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:54160
I20260616 19:56:55.048313 6992 raft_consensus.cc:606] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Received request to transfer leadership to TS 1c96bff2cfee441cad5e5acecd2bca1d
I20260616 19:56:55.111516 7116 raft_consensus.cc:993] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:56:55.112088 7131 raft_consensus.cc:1081] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Signalling peer 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:56:55.114243 7064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:40948
I20260616 19:56:55.114841 7064 raft_consensus.cc:493] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:56:55.115218 7064 raft_consensus.cc:3060] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.122506 7064 raft_consensus.cc:515] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.124641 7064 leader_election.cc:290] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 2 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:55.125869 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:55.126488 6992 raft_consensus.cc:3055] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:56:55.126808 6992 raft_consensus.cc:740] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:55.127017 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df344a41f1f445ab825428978e0e8c87" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:55.127640 6917 raft_consensus.cc:3060] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.127614 6992 consensus_queue.cc:260] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.129189 6992 raft_consensus.cc:3060] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.136071 6917 raft_consensus.cc:2468] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 2.
I20260616 19:56:55.136279 6992 raft_consensus.cc:2468] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 2.
I20260616 19:56:55.137600 7032 leader_election.cc:304] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:56:55.138505 7201 raft_consensus.cc:2804] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:56:55.139992 7201 raft_consensus.cc:697] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 LEADER]: Becoming Leader. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:56:55.140925 7201 consensus_queue.cc:237] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.150413 6814 catalog_manager.cc:5696] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d reported cstate change: term changed from 1 to 2, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131). New cstate: current_term: 2 leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: HEALTHY } } }
I20260616 19:56:55.214952 7201 raft_consensus.cc:993] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:56:55.215652 7201 raft_consensus.cc:1081] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:56:55.217902 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54146
I20260616 19:56:55.218591 6992 raft_consensus.cc:493] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:56:55.219018 6992 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.227473 6992 raft_consensus.cc:515] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.231896 7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:55.231693 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:55.232760 7064 raft_consensus.cc:3055] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:56:55.233003 6917 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.233263 7064 raft_consensus.cc:740] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:56:55.234387 7064 consensus_queue.cc:260] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.235591 7064 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.244820 6917 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:56:55.246127 6957 leader_election.cc:304] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:55.247100 7131 raft_consensus.cc:2804] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:56:55.247608 7131 raft_consensus.cc:697] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:55.248450 7131 consensus_queue.cc:237] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.248943 6992 leader_election.cc:290] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:55.251812 7064 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:56:55.258919 6814 catalog_manager.cc:5696] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 1 to 2, leader changed from 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:55.329242 7174 raft_consensus.cc:993] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:56:55.329749 7173 raft_consensus.cc:1081] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:56:55.331616 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54140
I20260616 19:56:55.332322 6992 raft_consensus.cc:493] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:56:55.332756 6992 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.340366 6992 raft_consensus.cc:515] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.343284 6992 leader_election.cc:290] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:55.345105 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:55.346134 6917 raft_consensus.cc:3055] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:56:55.346390 7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:55.346741 6917 raft_consensus.cc:740] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:55.347198 7064 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.348138 6917 consensus_queue.cc:260] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.351023 6917 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.361864 7064 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:56:55.363492 6956 leader_election.cc:304] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:56:55.364705 7131 raft_consensus.cc:2804] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:56:55.365324 7131 raft_consensus.cc:697] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:55.366384 7131 consensus_queue.cc:237] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.382511 6917 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:56:55.387735 7173 raft_consensus.cc:993] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:56:55.388346 7173 raft_consensus.cc:1081] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:56:55.387840 6813 catalog_manager.cc:5696] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 1 to 2, leader changed from 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:55.391328 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54140
I20260616 19:56:55.392822 6992 raft_consensus.cc:493] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:56:55.393319 6992 raft_consensus.cc:3060] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.402010 6992 raft_consensus.cc:515] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.405568 7116 raft_consensus.cc:993] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:56:55.405864 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:55.406538 6917 raft_consensus.cc:3055] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:56:55.406905 6917 raft_consensus.cc:740] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:55.406037 7116 raft_consensus.cc:1081] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Signalling peer 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:56:55.409297 6915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:35184
I20260616 19:56:55.409415 6917 consensus_queue.cc:260] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.409935 6915 raft_consensus.cc:493] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:56:55.410503 6915 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.410719 6917 raft_consensus.cc:3060] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.414573 6992 leader_election.cc:290] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:55.417035 7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb28b92f7f894493aa2479df8d5f14c9" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:55.417685 7064 raft_consensus.cc:3060] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.419239 6917 raft_consensus.cc:2468] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:56:55.420416 6957 leader_election.cc:304] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:55.421452 7116 raft_consensus.cc:2804] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:56:55.421229 6915 raft_consensus.cc:515] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.422617 7116 raft_consensus.cc:697] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:55.423460 7116 consensus_queue.cc:237] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.425041 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:56:55.425033 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:55.425719 7063 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.426050 6992 raft_consensus.cc:3055] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:56:55.426465 6992 raft_consensus.cc:740] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:55.427450 6992 consensus_queue.cc:260] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.428906 6992 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:55.431484 7064 raft_consensus.cc:2468] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:56:55.434607 7063 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 2.
I20260616 19:56:55.436228 6881 leader_election.cc:304] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:55.437032 6915 leader_election.cc:290] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 2 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:56:55.437211 7172 raft_consensus.cc:2804] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:56:55.437666 7172 raft_consensus.cc:697] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:55.438469 7172 consensus_queue.cc:237] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:55.442224 6992 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 2.
I20260616 19:56:55.443184 6814 catalog_manager.cc:5696] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 1 to 2, leader changed from 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:55.446960 6813 catalog_manager.cc:5696] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 1 to 2, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 2 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:55.607416 6992 raft_consensus.cc:1275] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.608771 7201 consensus_queue.cc:1048] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.618608 6915 raft_consensus.cc:1275] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.621045 7201 consensus_queue.cc:1048] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:56:55.776906 6915 raft_consensus.cc:1275] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.778296 7116 consensus_queue.cc:1048] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.788110 7063 raft_consensus.cc:1275] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.790128 7161 consensus_queue.cc:1048] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.858054 7063 raft_consensus.cc:1275] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.859309 7173 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.868209 6992 raft_consensus.cc:1275] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.872716 7173 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.914582 7063 raft_consensus.cc:1275] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.915938 7223 consensus_queue.cc:1048] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.925299 6915 raft_consensus.cc:1275] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.926721 7161 consensus_queue.cc:1048] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.946501 6915 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.947676 7161 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:55.960644 7063 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:55.962638 7161 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:58.076467 6782 auto_leader_rebalancer-test.cc:294] The leader distribution is
I20260616 19:56:58.077168 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:56:58.093092 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:51976
I20260616 19:56:58.093567 6992 raft_consensus.cc:606] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Received request to transfer leadership to TS 1c96bff2cfee441cad5e5acecd2bca1d
I20260616 19:56:58.103652 6915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c471229572c4a578e61219cdae70b45"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
mode: GRACEFUL
new_leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:60614
I20260616 19:56:58.104120 6915 raft_consensus.cc:606] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Received request to transfer leadership to TS 1c96bff2cfee441cad5e5acecd2bca1d
I20260616 19:56:58.105219 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260616 19:56:58.105605 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:56:58.182898 7242 raft_consensus.cc:993] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:56:58.183413 7246 raft_consensus.cc:1081] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Signalling peer 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:56:58.184928 7063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:40948
I20260616 19:56:58.185405 7063 raft_consensus.cc:493] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:56:58.185710 7063 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:56:58.190660 7063 raft_consensus.cc:515] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:58.192445 7063 leader_election.cc:290] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 3 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:58.193228 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:58.193485 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:58.193871 6915 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:56:58.194020 6992 raft_consensus.cc:3055] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:56:58.194360 6992 raft_consensus.cc:740] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:56:58.194998 6992 consensus_queue.cc:260] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:58.195904 6992 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:56:58.198685 6915 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 3.
I20260616 19:56:58.199695 7031 leader_election.cc:304] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:56:58.200403 7239 raft_consensus.cc:2804] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:56:58.200994 6992 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 3.
I20260616 19:56:58.201650 7239 raft_consensus.cc:697] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 LEADER]: Becoming Leader. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:56:58.202910 7239 consensus_queue.cc:237] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:56:58.209735 6813 catalog_manager.cc:5696] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d reported cstate change: term changed from 2 to 3, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131). New cstate: current_term: 3 leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: HEALTHY } } }
I20260616 19:56:58.374087 7240 raft_consensus.cc:993] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c: : Instructing follower 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:56:58.374527 7253 raft_consensus.cc:1081] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Signalling peer 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:56:58.376048 7063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c471229572c4a578e61219cdae70b45"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:40960
I20260616 19:56:58.376564 7063 raft_consensus.cc:493] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:56:58.376873 7063 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:58.381803 7063 raft_consensus.cc:515] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:58.383327 7063 leader_election.cc:290] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 2 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:56:58.384050 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:56:58.384311 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:56:58.384639 6915 raft_consensus.cc:3055] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:56:58.384832 6992 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:58.384994 6915 raft_consensus.cc:740] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:56:58.385804 6915 consensus_queue.cc:260] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:58.386961 6915 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:56:58.391558 6992 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 2.
I20260616 19:56:58.392675 7032 leader_election.cc:304] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:56:58.393364 7239 raft_consensus.cc:2804] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:56:58.393456 6915 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 2.
I20260616 19:56:58.393883 7239 raft_consensus.cc:697] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 LEADER]: Becoming Leader. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:56:58.394682 7239 consensus_queue.cc:237] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:56:58.401669 6813 catalog_manager.cc:5696] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d reported cstate change: term changed from 1 to 2, leader changed from 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129) to 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131). New cstate: current_term: 2 leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260616 19:56:58.666668 6992 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:56:58.667819 7239 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:56:58.676379 6915 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:56:58.678464 7247 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:56:58.789754 6992 raft_consensus.cc:1275] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:58.790916 7247 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:58.798954 6915 raft_consensus.cc:1275] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:56:58.800606 7257 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:56:59.106508 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:56:59.111548 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:56:59.111970 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:00.113096 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:00.118467 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:00.118868 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:01.119967 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:01.124977 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:01.125437 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:02.126380 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:02.131814 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:02.132261 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:03.133212 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:03.138383 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:03.138792 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:04.177973 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:04.182722 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:04.183157 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260616 19:57:05.044466 7013 debug-util.cc:398] Leaking SignalData structure 0x7b08001b0600 after lost signal to thread 6785
W20260616 19:57:05.045153 7013 debug-util.cc:398] Leaking SignalData structure 0x7b08001ca2e0 after lost signal to thread 6848
W20260616 19:57:05.045845 7013 debug-util.cc:398] Leaking SignalData structure 0x7b0800083bc0 after lost signal to thread 6941
W20260616 19:57:05.046473 7013 debug-util.cc:398] Leaking SignalData structure 0x7b08000f08a0 after lost signal to thread 7016
W20260616 19:57:05.047073 7013 debug-util.cc:398] Leaking SignalData structure 0x7b08001c19e0 after lost signal to thread 7090
I20260616 19:57:05.184098 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:05.189163 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:05.189572 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:06.190541 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:06.195333 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:06.195801 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:07.196734 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:07.201478 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:07.201910 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:08.229156 6782 auto_leader_rebalancer-test.cc:313] The leader distribution is
I20260616 19:57:08.233309 7063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c471229572c4a578e61219cdae70b45"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:40964
I20260616 19:57:08.233947 7063 raft_consensus.cc:606] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:57:08.238569 6917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:35194
I20260616 19:57:08.238989 6917 raft_consensus.cc:606] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:57:08.243371 6917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:35194
I20260616 19:57:08.243788 6917 raft_consensus.cc:606] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:57:08.248353 6917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:35194
I20260616 19:57:08.248777 6917 raft_consensus.cc:606] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:57:08.254660 7273 raft_consensus.cc:993] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.255283 7269 raft_consensus.cc:1081] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.255903 7063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
mode: GRACEFUL
new_leader_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:40964
I20260616 19:57:08.256480 7063 raft_consensus.cc:606] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 LEADER]: Received request to transfer leadership to TS 40a2a770d461463e9a88835aab4e6c73
I20260616 19:57:08.257144 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54140
I20260616 19:57:08.257705 6992 raft_consensus.cc:493] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:08.258062 6992 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:08.264539 6992 raft_consensus.cc:515] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.267269 6992 leader_election.cc:290] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 3 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:08.268211 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:57:08.268345 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:08.269006 7063 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:08.268997 6917 raft_consensus.cc:3055] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:08.269546 6917 raft_consensus.cc:740] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:57:08.270416 6917 consensus_queue.cc:260] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.271826 6917 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:08.276070 7063 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 3.
I20260616 19:57:08.277527 6956 leader_election.cc:304] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:57:08.278401 7272 raft_consensus.cc:2804] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:08.278539 6917 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 3.
I20260616 19:57:08.279609 7272 raft_consensus.cc:697] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:08.280515 7272 consensus_queue.cc:237] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.290203 6814 catalog_manager.cc:5696] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 2 to 3, leader changed from 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:08.338089 7273 raft_consensus.cc:993] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.338518 7273 raft_consensus.cc:1081] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.339887 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54140
I20260616 19:57:08.340303 6992 raft_consensus.cc:493] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:08.340611 6992 raft_consensus.cc:3060] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:08.344586 6992 raft_consensus.cc:515] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.345968 6992 leader_election.cc:290] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:08.346801 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:57:08.346909 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97b4eb44988f4d26beff2c9420faec08" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:08.347417 6917 raft_consensus.cc:3055] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:08.347510 7063 raft_consensus.cc:3060] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:08.347782 6917 raft_consensus.cc:740] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:57:08.348523 6917 consensus_queue.cc:260] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.349601 6917 raft_consensus.cc:3060] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:08.353710 7063 raft_consensus.cc:2468] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:57:08.354851 6956 leader_election.cc:304] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:57:08.355512 6917 raft_consensus.cc:2468] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:57:08.355559 7272 raft_consensus.cc:2804] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:08.356101 7272 raft_consensus.cc:697] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:08.357044 7272 consensus_queue.cc:237] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.364394 6814 catalog_manager.cc:5696] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 1 to 2, leader changed from 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:08.412689 7281 raft_consensus.cc:993] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.413275 7268 raft_consensus.cc:1081] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.414554 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54146
I20260616 19:57:08.414986 6992 raft_consensus.cc:493] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:08.415263 6992 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:08.419297 6992 raft_consensus.cc:515] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.420749 6992 leader_election.cc:290] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 4 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:08.421643 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:57:08.421663 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:08.422132 6917 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:08.422302 7063 raft_consensus.cc:3055] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:57:08.422638 7063 raft_consensus.cc:740] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:57:08.423249 7063 consensus_queue.cc:260] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.424129 7063 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:08.426339 6917 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 4.
I20260616 19:57:08.427321 6957 leader_election.cc:304] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:57:08.428004 7272 raft_consensus.cc:2804] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:57:08.428467 7063 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 4.
I20260616 19:57:08.428582 7272 raft_consensus.cc:697] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:08.429478 7272 consensus_queue.cc:237] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.436188 6814 catalog_manager.cc:5696] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 3 to 4, leader changed from 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:08.568568 7269 raft_consensus.cc:993] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.569123 7273 raft_consensus.cc:1081] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.570526 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54140
I20260616 19:57:08.571004 6992 raft_consensus.cc:493] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:08.571293 6992 raft_consensus.cc:3060] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:08.576210 6992 raft_consensus.cc:515] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.577620 6992 leader_election.cc:290] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:08.578325 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:57:08.578524 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b4ba1bad40148d08f3197e703269a9b" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:08.578827 6917 raft_consensus.cc:3055] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:08.579142 7063 raft_consensus.cc:3060] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:08.579152 6917 raft_consensus.cc:740] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:57:08.579998 6917 consensus_queue.cc:260] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.580960 6917 raft_consensus.cc:3060] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:08.583562 7063 raft_consensus.cc:2468] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:57:08.584573 6956 leader_election.cc:304] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 40a2a770d461463e9a88835aab4e6c73; no voters:
I20260616 19:57:08.585240 7272 raft_consensus.cc:2804] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:08.585729 7272 raft_consensus.cc:697] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:08.585402 6917 raft_consensus.cc:2468] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 2.
I20260616 19:57:08.586577 7272 consensus_queue.cc:237] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:08.589262 7281 raft_consensus.cc:993] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d: : Instructing follower 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.589802 7268 raft_consensus.cc:1081] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 LEADER]: Signalling peer 40a2a770d461463e9a88835aab4e6c73 to start an election
I20260616 19:57:08.591560 6992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c471229572c4a578e61219cdae70b45"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
from {username='slave'} at 127.0.0.1:54146
I20260616 19:57:08.591958 6992 raft_consensus.cc:493] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:08.592248 6992 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:08.596508 6814 catalog_manager.cc:5696] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 1 to 2, leader changed from 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:08.597656 6992 raft_consensus.cc:515] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:57:08.599658 6992 leader_election.cc:290] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 3 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:08.600047 6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:57:08.600142 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "40a2a770d461463e9a88835aab4e6c73" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:08.600733 6917 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:08.600795 7063 raft_consensus.cc:3055] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:08.601244 7063 raft_consensus.cc:740] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:57:08.601979 7063 consensus_queue.cc:260] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:57:08.602928 7063 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:08.606123 6917 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 3.
I20260616 19:57:08.607025 6957 leader_election.cc:304] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a2a770d461463e9a88835aab4e6c73, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:57:08.607798 7272 raft_consensus.cc:2804] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:08.608306 7272 raft_consensus.cc:697] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Becoming Leader. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:08.608312 7063 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a2a770d461463e9a88835aab4e6c73 in term 3.
I20260616 19:57:08.609299 7272 consensus_queue.cc:237] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:57:08.616200 6814 catalog_manager.cc:5696] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 reported cstate change: term changed from 2 to 3, leader changed from 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131) to 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "40a2a770d461463e9a88835aab4e6c73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:08.710826 7063 raft_consensus.cc:1275] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:08.711896 7272 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:08.721299 6917 raft_consensus.cc:1275] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:08.723372 7287 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:57:08.841517 6917 raft_consensus.cc:1275] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:08.841508 7063 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:08.842849 7287 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:08.842959 7272 consensus_queue.cc:1048] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:08.855746 7064 raft_consensus.cc:1275] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:08.859737 6917 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:08.859582 7287 consensus_queue.cc:1048] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:08.861222 7272 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:09.101055 6917 raft_consensus.cc:1275] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:09.102161 7287 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:09.114642 7064 raft_consensus.cc:1275] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:09.116323 7287 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:57:09.129707 7063 raft_consensus.cc:1275] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:09.131290 7287 consensus_queue.cc:1048] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:57:09.142756 6917 raft_consensus.cc:1275] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Refusing update from remote peer 40a2a770d461463e9a88835aab4e6c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:09.144299 7287 consensus_queue.cc:1048] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:11.293388 6782 auto_leader_rebalancer-test.cc:328] The leader distribution is
I20260616 19:57:11.294221 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:11.301869 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:51976
I20260616 19:57:11.302433 6992 raft_consensus.cc:606] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 LEADER]: Received request to transfer leadership to TS 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:57:11.304677 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:51976
I20260616 19:57:11.305058 6992 raft_consensus.cc:606] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Received request to transfer leadership to TS 1c96bff2cfee441cad5e5acecd2bca1d
I20260616 19:57:11.306950 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c471229572c4a578e61219cdae70b45"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:51976
I20260616 19:57:11.307299 6992 raft_consensus.cc:606] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Received request to transfer leadership to TS 1c96bff2cfee441cad5e5acecd2bca1d
I20260616 19:57:11.309373 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:51976
I20260616 19:57:11.309800 6992 raft_consensus.cc:606] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Received request to transfer leadership to TS 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:57:11.311823 6992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b"
dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
mode: GRACEFUL
new_leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:51976
I20260616 19:57:11.312256 6992 raft_consensus.cc:606] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Received request to transfer leadership to TS 4cac913e39174d1c9fd124a69d6e4f6c
I20260616 19:57:11.313270 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:57:11.313661 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:11.426626 7303 raft_consensus.cc:993] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:57:11.427132 7287 raft_consensus.cc:1081] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Signalling peer 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:57:11.428644 6917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:35184
I20260616 19:57:11.429121 6917 raft_consensus.cc:493] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:11.429438 6917 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:11.434136 6917 raft_consensus.cc:515] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.436003 6917 leader_election.cc:290] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 4 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:11.437094 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:57:11.437350 7063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a7d4ad6e3d4d92a83f12fd31ece99d" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:11.437744 6992 raft_consensus.cc:3055] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:57:11.437861 7063 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:11.438014 6992 raft_consensus.cc:740] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:11.438810 6992 consensus_queue.cc:260] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.439731 6992 raft_consensus.cc:3060] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:11.442337 7063 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 4.
I20260616 19:57:11.443490 6881 leader_election.cc:304] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:57:11.444339 6992 raft_consensus.cc:2468] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 4.
I20260616 19:57:11.445240 7304 raft_consensus.cc:2804] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:57:11.446190 7304 raft_consensus.cc:697] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:57:11.447062 7304 consensus_queue.cc:237] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.453539 6814 catalog_manager.cc:5696] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 3 to 4, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 4 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:11.565546 7287 raft_consensus.cc:993] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:57:11.566018 7303 raft_consensus.cc:1081] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Signalling peer 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:57:11.567662 7303 raft_consensus.cc:993] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:57:11.567761 6915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:35184
I20260616 19:57:11.568265 7311 raft_consensus.cc:1081] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Signalling peer 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:57:11.568295 6915 raft_consensus.cc:493] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:11.568696 6915 raft_consensus.cc:3060] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:11.569912 7063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c471229572c4a578e61219cdae70b45"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:40948
I20260616 19:57:11.570364 7063 raft_consensus.cc:493] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:11.570611 7063 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:11.573616 6915 raft_consensus.cc:515] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.574615 7063 raft_consensus.cc:515] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:57:11.575081 6915 leader_election.cc:290] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 2 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:11.575989 6992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:57:11.576290 7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83fd0e6f6b8e4a34b9f92ffa3f38f63b" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:11.576530 6992 raft_consensus.cc:3055] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:11.576833 6992 raft_consensus.cc:740] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:11.576884 7064 raft_consensus.cc:3060] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:11.577069 7063 leader_election.cc:290] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 4 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:57:11.577621 6992 consensus_queue.cc:260] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.578424 6990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:57:11.578896 6992 raft_consensus.cc:3060] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:11.578991 6990 raft_consensus.cc:3055] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:57:11.579322 6990 raft_consensus.cc:740] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:11.579471 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c471229572c4a578e61219cdae70b45" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:57:11.580113 6915 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:11.580108 6990 consensus_queue.cc:260] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:57:11.581410 6990 raft_consensus.cc:3060] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:11.583753 7064 raft_consensus.cc:2468] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 2.
I20260616 19:57:11.584764 6881 leader_election.cc:304] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:57:11.585412 7304 raft_consensus.cc:2804] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:11.586603 7304 raft_consensus.cc:697] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:57:11.587519 6915 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 4.
I20260616 19:57:11.587833 6992 raft_consensus.cc:2468] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 2.
I20260616 19:57:11.587661 7304 consensus_queue.cc:237] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.588753 6990 raft_consensus.cc:2468] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 4.
I20260616 19:57:11.588881 7031 leader_election.cc:304] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:57:11.589694 7268 raft_consensus.cc:2804] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:57:11.590945 7268 raft_consensus.cc:697] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 LEADER]: Becoming Leader. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:57:11.591811 7268 consensus_queue.cc:237] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } }
I20260616 19:57:11.596537 6814 catalog_manager.cc:5696] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 1 to 2, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 2 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:11.605511 6811 catalog_manager.cc:5696] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d reported cstate change: term changed from 3 to 4, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131). New cstate: current_term: 4 leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:11.612360 7311 raft_consensus.cc:993] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:57:11.612828 7303 raft_consensus.cc:1081] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Signalling peer 1c96bff2cfee441cad5e5acecd2bca1d to start an election
I20260616 19:57:11.614197 7064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83"
dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
from {username='slave'} at 127.0.0.1:40948
I20260616 19:57:11.614603 7064 raft_consensus.cc:493] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:11.614823 7064 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:11.619006 7064 raft_consensus.cc:515] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.620504 7064 leader_election.cc:290] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 3 election: Requested vote from peers 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747), 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141)
I20260616 19:57:11.621322 6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
I20260616 19:57:11.621529 6990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1393cafa166044e18db7522c4ee84a83" candidate_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:57:11.621832 6915 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:11.622057 6990 raft_consensus.cc:3055] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:11.622385 6990 raft_consensus.cc:740] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:11.622992 6990 consensus_queue.cc:260] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.623893 6990 raft_consensus.cc:3060] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:11.626312 6915 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 3.
I20260616 19:57:11.627266 7031 leader_election.cc:304] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:57:11.627976 7268 raft_consensus.cc:2804] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:11.628435 7268 raft_consensus.cc:697] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 LEADER]: Becoming Leader. State: Replica: 1c96bff2cfee441cad5e5acecd2bca1d, State: Running, Role: LEADER
I20260616 19:57:11.628979 6990 raft_consensus.cc:2468] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c96bff2cfee441cad5e5acecd2bca1d in term 3.
I20260616 19:57:11.629236 7268 consensus_queue.cc:237] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.635622 6811 catalog_manager.cc:5696] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d reported cstate change: term changed from 2 to 3, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131). New cstate: current_term: 3 leader_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:11.800771 7311 raft_consensus.cc:993] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: : Instructing follower 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:57:11.801185 7303 raft_consensus.cc:1081] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 LEADER]: Signalling peer 4cac913e39174d1c9fd124a69d6e4f6c to start an election
I20260616 19:57:11.802404 6915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05d07be714a74ac680a9502b788081c1"
dest_uuid: "4cac913e39174d1c9fd124a69d6e4f6c"
from {username='slave'} at 127.0.0.1:35184
I20260616 19:57:11.802834 6915 raft_consensus.cc:493] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:11.803143 6915 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:57:11.807246 6915 raft_consensus.cc:515] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.808691 6915 leader_election.cc:290] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 5 election: Requested vote from peers 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141), 1c96bff2cfee441cad5e5acecd2bca1d (127.6.159.131:36707)
I20260616 19:57:11.809561 6990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "40a2a770d461463e9a88835aab4e6c73"
I20260616 19:57:11.809671 7064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d07be714a74ac680a9502b788081c1" candidate_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "1c96bff2cfee441cad5e5acecd2bca1d"
I20260616 19:57:11.810043 6990 raft_consensus.cc:3055] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:57:11.810169 7064 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:57:11.810338 6990 raft_consensus.cc:740] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 40a2a770d461463e9a88835aab4e6c73, State: Running, Role: LEADER
I20260616 19:57:11.811094 6990 consensus_queue.cc:260] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.812017 6990 raft_consensus.cc:3060] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:57:11.814289 7064 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 5.
I20260616 19:57:11.815201 6881 leader_election.cc:304] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c96bff2cfee441cad5e5acecd2bca1d, 4cac913e39174d1c9fd124a69d6e4f6c; no voters:
I20260616 19:57:11.815912 7304 raft_consensus.cc:2804] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:57:11.816335 7304 raft_consensus.cc:697] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 5 LEADER]: Becoming Leader. State: Replica: 4cac913e39174d1c9fd124a69d6e4f6c, State: Running, Role: LEADER
I20260616 19:57:11.816810 6990 raft_consensus.cc:2468] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cac913e39174d1c9fd124a69d6e4f6c in term 5.
I20260616 19:57:11.817067 7304 consensus_queue.cc:237] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } }
I20260616 19:57:11.823982 6811 catalog_manager.cc:5696] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c reported cstate change: term changed from 4 to 5, leader changed from 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130) to 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129). New cstate: current_term: 5 leader_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:11.874571 6990 raft_consensus.cc:1275] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:11.875824 7304 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:11.885957 7064 raft_consensus.cc:1275] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:11.888267 7304 consensus_queue.cc:1048] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:11.976408 6990 raft_consensus.cc:1275] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:11.977646 7268 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:11.979387 6990 raft_consensus.cc:1275] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:11.980651 7304 consensus_queue.cc:1048] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:11.987838 6915 raft_consensus.cc:1275] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:11.992501 7268 consensus_queue.cc:1048] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:12.004981 7064 raft_consensus.cc:1275] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:12.006997 7304 consensus_queue.cc:1048] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:57:12.212832 6917 raft_consensus.cc:1275] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:12.214052 7325 consensus_queue.cc:1048] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac913e39174d1c9fd124a69d6e4f6c" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 45747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:12.222921 6990 raft_consensus.cc:1275] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Refusing update from remote peer 1c96bff2cfee441cad5e5acecd2bca1d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:12.224321 7325 consensus_queue.cc:1048] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:12.230602 6992 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 5 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:57:12.231833 7327 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a2a770d461463e9a88835aab4e6c73" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:57:12.240239 7064 raft_consensus.cc:1275] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 5 FOLLOWER]: Refusing update from remote peer 4cac913e39174d1c9fd124a69d6e4f6c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:57:12.241878 7305 consensus_queue.cc:1048] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c96bff2cfee441cad5e5acecd2bca1d" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:57:12.314797 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:12.319725 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:12.320088 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:13.321156 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:13.326027 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:13.326450 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:14.327474 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:14.332592 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:14.332994 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:15.333992 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:15.339054 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:15.339413 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:16.340466 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:16.345201 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:16.345534 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:17.346520 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:17.352422 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:17.352936 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:18.354067 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:18.359118 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:18.359493 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:19.360445 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:19.365096 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:19.365437 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:20.366600 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:20.371932 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:20.372328 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:21.399672 6782 auto_leader_rebalancer-test.cc:340] The leader distribution is
I20260616 19:57:21.400204 6782 tablet_server.cc:179] TabletServer@127.6.159.129:0 shutting down...
I20260616 19:57:21.426203 6782 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260616 19:57:21.426859 6782 tablet_replica.cc:333] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.427645 6782 raft_consensus.cc:2243] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 5 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.428747 6782 raft_consensus.cc:2272] T 05d07be714a74ac680a9502b788081c1 P 4cac913e39174d1c9fd124a69d6e4f6c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.431034 6782 tablet_replica.cc:333] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.431586 6782 raft_consensus.cc:2243] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.432027 6782 raft_consensus.cc:2272] T 7b4ba1bad40148d08f3197e703269a9b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.433718 6782 tablet_replica.cc:333] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.434167 6782 raft_consensus.cc:2243] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.434545 6782 raft_consensus.cc:2272] T 1393cafa166044e18db7522c4ee84a83 P 4cac913e39174d1c9fd124a69d6e4f6c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.436045 6782 tablet_replica.cc:333] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.436570 6782 raft_consensus.cc:2243] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.437233 6782 raft_consensus.cc:2272] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.438755 6782 tablet_replica.cc:333] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.439174 6782 raft_consensus.cc:2243] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.439520 6782 raft_consensus.cc:2272] T 3c471229572c4a578e61219cdae70b45 P 4cac913e39174d1c9fd124a69d6e4f6c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.441030 6782 tablet_replica.cc:333] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.441501 6782 raft_consensus.cc:2243] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.441872 6782 raft_consensus.cc:2272] T 97b4eb44988f4d26beff2c9420faec08 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.443745 6782 tablet_replica.cc:333] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.444144 6782 raft_consensus.cc:2243] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.444865 6782 raft_consensus.cc:2272] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.446375 6782 tablet_replica.cc:333] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.446753 6782 raft_consensus.cc:2243] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.447074 6782 raft_consensus.cc:2272] T df344a41f1f445ab825428978e0e8c87 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.448500 6782 tablet_replica.cc:333] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c: stopping tablet replica
I20260616 19:57:21.448869 6782 raft_consensus.cc:2243] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.449211 6782 raft_consensus.cc:2272] T eb28b92f7f894493aa2479df8d5f14c9 P 4cac913e39174d1c9fd124a69d6e4f6c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.536577 6782 tablet_server.cc:196] TabletServer@127.6.159.129:0 shutdown complete.
I20260616 19:57:21.554095 6782 tablet_server.cc:179] TabletServer@127.6.159.130:0 shutting down...
W20260616 19:57:21.564646 7031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.159.129:45747: connect: Connection refused (error 111)
W20260616 19:57:21.572017 7031 consensus_peers.cc:597] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d -> Peer 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747): Couldn't send request to peer 4cac913e39174d1c9fd124a69d6e4f6c. Status: Network error: Client connection negotiation failed: client connection to 127.6.159.129:45747: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260616 19:57:21.585870 6782 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260616 19:57:21.586548 6782 tablet_replica.cc:333] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.587507 6782 raft_consensus.cc:2243] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.588596 6782 raft_consensus.cc:2272] T 7b4ba1bad40148d08f3197e703269a9b P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.592087 6782 tablet_replica.cc:333] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.592742 6782 raft_consensus.cc:2243] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.593662 6782 raft_consensus.cc:2272] T eb28b92f7f894493aa2479df8d5f14c9 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.595920 6782 tablet_replica.cc:333] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
W20260616 19:57:21.595862 7032 consensus_peers.cc:597] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d -> Peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Couldn't send request to peer 40a2a770d461463e9a88835aab4e6c73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:57:21.596629 6782 raft_consensus.cc:2243] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.597051 6782 raft_consensus.cc:2272] T 3c471229572c4a578e61219cdae70b45 P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.598634 6782 tablet_replica.cc:333] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.599087 6782 raft_consensus.cc:2243] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.599452 6782 raft_consensus.cc:2272] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 40a2a770d461463e9a88835aab4e6c73 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.601143 6782 tablet_replica.cc:333] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.601598 6782 raft_consensus.cc:2243] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.602301 6782 raft_consensus.cc:2272] T 97b4eb44988f4d26beff2c9420faec08 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.603924 6782 tablet_replica.cc:333] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.604460 6782 raft_consensus.cc:2243] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.604920 6782 raft_consensus.cc:2272] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.606529 6782 tablet_replica.cc:333] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.606935 6782 raft_consensus.cc:2243] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.607270 6782 raft_consensus.cc:2272] T 05d07be714a74ac680a9502b788081c1 P 40a2a770d461463e9a88835aab4e6c73 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.608887 6782 tablet_replica.cc:333] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.609266 6782 raft_consensus.cc:2243] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.609620 6782 raft_consensus.cc:2272] T df344a41f1f445ab825428978e0e8c87 P 40a2a770d461463e9a88835aab4e6c73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.611117 6782 tablet_replica.cc:333] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73: stopping tablet replica
I20260616 19:57:21.611516 6782 raft_consensus.cc:2243] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.611852 6782 raft_consensus.cc:2272] T 1393cafa166044e18db7522c4ee84a83 P 40a2a770d461463e9a88835aab4e6c73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.664397 6782 tablet_server.cc:196] TabletServer@127.6.159.130:0 shutdown complete.
I20260616 19:57:21.682927 6782 tablet_server.cc:179] TabletServer@127.6.159.131:0 shutting down...
W20260616 19:57:21.707188 7032 consensus_peers.cc:597] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d -> Peer 40a2a770d461463e9a88835aab4e6c73 (127.6.159.130:42141): Couldn't send request to peer 40a2a770d461463e9a88835aab4e6c73. Status: Network error: Client connection negotiation failed: client connection to 127.6.159.130:42141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260616 19:57:21.715250 7031 consensus_peers.cc:597] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d -> Peer 4cac913e39174d1c9fd124a69d6e4f6c (127.6.159.129:45747): Couldn't send request to peer 4cac913e39174d1c9fd124a69d6e4f6c. Status: Network error: Client connection negotiation failed: client connection to 127.6.159.129:45747: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260616 19:57:21.715591 6782 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260616 19:57:21.716342 6782 tablet_replica.cc:333] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.716964 6782 raft_consensus.cc:2243] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.717507 6782 raft_consensus.cc:2272] T 05d07be714a74ac680a9502b788081c1 P 1c96bff2cfee441cad5e5acecd2bca1d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.719209 6782 tablet_replica.cc:333] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.719723 6782 raft_consensus.cc:2243] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.720227 6782 raft_consensus.cc:2272] T 97b4eb44988f4d26beff2c9420faec08 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.721864 6782 tablet_replica.cc:333] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.722345 6782 raft_consensus.cc:2243] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.722726 6782 raft_consensus.cc:2272] T 83fd0e6f6b8e4a34b9f92ffa3f38f63b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.724411 6782 tablet_replica.cc:333] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.724872 6782 raft_consensus.cc:2243] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.725256 6782 raft_consensus.cc:2272] T 77a7d4ad6e3d4d92a83f12fd31ece99d P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.726776 6782 tablet_replica.cc:333] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.727254 6782 raft_consensus.cc:2243] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.728003 6782 raft_consensus.cc:2272] T 3c471229572c4a578e61219cdae70b45 P 1c96bff2cfee441cad5e5acecd2bca1d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.729595 6782 tablet_replica.cc:333] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.730054 6782 raft_consensus.cc:2243] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.730468 6782 raft_consensus.cc:2272] T eb28b92f7f894493aa2479df8d5f14c9 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.731966 6782 tablet_replica.cc:333] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.732479 6782 raft_consensus.cc:2243] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.733188 6782 raft_consensus.cc:2272] T df344a41f1f445ab825428978e0e8c87 P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.734721 6782 tablet_replica.cc:333] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.735165 6782 raft_consensus.cc:2243] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.735872 6782 raft_consensus.cc:2272] T 1393cafa166044e18db7522c4ee84a83 P 1c96bff2cfee441cad5e5acecd2bca1d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.737406 6782 tablet_replica.cc:333] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d: stopping tablet replica
I20260616 19:57:21.737854 6782 raft_consensus.cc:2243] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:21.738233 6782 raft_consensus.cc:2272] T 7b4ba1bad40148d08f3197e703269a9b P 1c96bff2cfee441cad5e5acecd2bca1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.764568 6782 tablet_server.cc:196] TabletServer@127.6.159.131:0 shutdown complete.
I20260616 19:57:21.781771 6782 master.cc:562] Master@127.6.159.190:36419 shutting down...
I20260616 19:57:21.897881 6782 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 1 LEADER]: Raft consensus shutting down.
I20260616 19:57:21.898535 6782 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:21.898895 6782 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1784bd25ee1c42c6891d1a6d818bb721: stopping tablet replica
I20260616 19:57:21.918756 6782 master.cc:584] Master@127.6.159.190:36419 shutdown complete.
1c96bff2cfee441cad5e5acecd2bca1d 1
40a2a770d461463e9a88835aab4e6c73 4
4cac913e39174d1c9fd124a69d6e4f6c 4
1c96bff2cfee441cad5e5acecd2bca1d 3
40a2a770d461463e9a88835aab4e6c73 3
4cac913e39174d1c9fd124a69d6e4f6c 3
1c96bff2cfee441cad5e5acecd2bca1d 1
40a2a770d461463e9a88835aab4e6c73 8
4cac913e39174d1c9fd124a69d6e4f6c 0
1c96bff2cfee441cad5e5acecd2bca1d 3
40a2a770d461463e9a88835aab4e6c73 3
4cac913e39174d1c9fd124a69d6e4f6c 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (32362 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260616 19:57:21.960952 6782 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.159.190:37049
I20260616 19:57:21.962109 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:21.967881 7339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:21.967839 7338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:21.969242 6782 server_base.cc:1061] running on GCE node
W20260616 19:57:21.969187 7341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:21.970530 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:21.970721 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:21.970839 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639841970828 us; error 0 us; skew 500 ppm
I20260616 19:57:21.971294 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:21.973788 6782 webserver.cc:533] Webserver started at http://127.6.159.190:41647/ using document root <none> and password file <none>
I20260616 19:57:21.974236 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:21.974396 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:21.974606 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:21.975682 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/master-0-root/instance:
uuid: "9539693897dc4bb9982fcca5e1267193"
format_stamp: "Formatted at 2026-06-16 19:57:21 on dist-test-slave-1c0m"
I20260616 19:57:21.980003 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260616 19:57:21.983217 7346 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:21.984014 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260616 19:57:21.984354 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/master-0-root
uuid: "9539693897dc4bb9982fcca5e1267193"
format_stamp: "Formatted at 2026-06-16 19:57:21 on dist-test-slave-1c0m"
I20260616 19:57:21.984650 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:22.006029 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:22.007290 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:22.044997 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.190:37049
I20260616 19:57:22.045084 7397 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.190:37049 every 8 connection(s)
I20260616 19:57:22.048978 7398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.058535 7398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193: Bootstrap starting.
I20260616 19:57:22.063122 7398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.067344 7398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193: No bootstrap required, opened a new log
I20260616 19:57:22.069546 7398 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9539693897dc4bb9982fcca5e1267193" member_type: VOTER }
I20260616 19:57:22.069978 7398 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.070153 7398 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9539693897dc4bb9982fcca5e1267193, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.070627 7398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9539693897dc4bb9982fcca5e1267193" member_type: VOTER }
I20260616 19:57:22.071053 7398 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260616 19:57:22.071241 7398 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260616 19:57:22.071477 7398 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:22.075520 7398 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9539693897dc4bb9982fcca5e1267193" member_type: VOTER }
I20260616 19:57:22.076009 7398 leader_election.cc:304] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9539693897dc4bb9982fcca5e1267193; no voters:
I20260616 19:57:22.077095 7398 leader_election.cc:290] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260616 19:57:22.077512 7401 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:22.078852 7401 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 1 LEADER]: Becoming Leader. State: Replica: 9539693897dc4bb9982fcca5e1267193, State: Running, Role: LEADER
I20260616 19:57:22.079509 7401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9539693897dc4bb9982fcca5e1267193" member_type: VOTER }
I20260616 19:57:22.080098 7398 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [sys.catalog]: configured and running, proceeding with master startup.
I20260616 19:57:22.084769 7402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9539693897dc4bb9982fcca5e1267193" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9539693897dc4bb9982fcca5e1267193" member_type: VOTER } }
I20260616 19:57:22.085246 7402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [sys.catalog]: This master's current role is: LEADER
I20260616 19:57:22.085108 7403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9539693897dc4bb9982fcca5e1267193. Latest consensus state: current_term: 1 leader_uuid: "9539693897dc4bb9982fcca5e1267193" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9539693897dc4bb9982fcca5e1267193" member_type: VOTER } }
I20260616 19:57:22.085821 7403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [sys.catalog]: This master's current role is: LEADER
I20260616 19:57:22.091873 7409 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
I20260616 19:57:22.098604 7409 catalog_manager.cc:1519] Initializing Kudu cluster ID...
I20260616 19:57:22.099675 6782 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260616 19:57:22.107440 7409 catalog_manager.cc:1382] Generated new cluster ID: a88b5ff5045749eda53a7cd57b38d0cf
I20260616 19:57:22.107693 7409 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
I20260616 19:57:22.121826 7409 catalog_manager.cc:1405] Generated new certificate authority record
I20260616 19:57:22.122991 7409 catalog_manager.cc:1539] Loading token signing keys...
I20260616 19:57:22.138972 7409 catalog_manager.cc:6069] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193: Generated new TSK 0
I20260616 19:57:22.139536 7409 catalog_manager.cc:1549] Initializing in-progress tserver states...
I20260616 19:57:22.166612 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:22.172495 7419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:22.173656 7420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:22.175263 6782 server_base.cc:1061] running on GCE node
W20260616 19:57:22.175997 7422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:22.176934 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:22.177110 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:22.177228 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639842177217 us; error 0 us; skew 500 ppm
I20260616 19:57:22.177685 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:22.179924 6782 webserver.cc:533] Webserver started at http://127.6.159.129:37101/ using document root <none> and password file <none>
I20260616 19:57:22.180419 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:22.180574 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:22.180778 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:22.181799 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-0-root/instance:
uuid: "9f84cb1d073a4fd79a5309423cb71258"
format_stamp: "Formatted at 2026-06-16 19:57:22 on dist-test-slave-1c0m"
I20260616 19:57:22.186196 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260616 19:57:22.189289 7427 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.189993 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260616 19:57:22.190228 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-0-root
uuid: "9f84cb1d073a4fd79a5309423cb71258"
format_stamp: "Formatted at 2026-06-16 19:57:22 on dist-test-slave-1c0m"
I20260616 19:57:22.190490 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:22.223145 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:22.224345 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:22.225818 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:57:22.227941 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:57:22.228108 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.228425 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:57:22.228596 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.266630 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.129:43255
I20260616 19:57:22.266717 7489 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.129:43255 every 8 connection(s)
I20260616 19:57:22.271536 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:22.278788 7494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:22.278873 7495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:22.282425 6782 server_base.cc:1061] running on GCE node
W20260616 19:57:22.283151 7497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:22.284125 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:22.284379 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:22.284505 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639842284494 us; error 0 us; skew 500 ppm
I20260616 19:57:22.284940 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:22.287774 6782 webserver.cc:533] Webserver started at http://127.6.159.130:34531/ using document root <none> and password file <none>
I20260616 19:57:22.288496 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:22.288754 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:22.289096 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:22.289440 7490 heartbeater.cc:344] Connected to a master server at 127.6.159.190:37049
I20260616 19:57:22.289836 7490 heartbeater.cc:461] Registering TS with master...
I20260616 19:57:22.290764 7490 heartbeater.cc:507] Master 127.6.159.190:37049 requested a full tablet report, sending...
I20260616 19:57:22.290699 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-1-root/instance:
uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
format_stamp: "Formatted at 2026-06-16 19:57:22 on dist-test-slave-1c0m"
I20260616 19:57:22.293237 7363 ts_manager.cc:194] Registered new tserver with Master: 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255)
I20260616 19:57:22.295608 7363 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37590
I20260616 19:57:22.296931 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260616 19:57:22.300742 7502 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.301491 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260616 19:57:22.301738 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-1-root
uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
format_stamp: "Formatted at 2026-06-16 19:57:22 on dist-test-slave-1c0m"
I20260616 19:57:22.301983 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:22.344396 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:22.345526 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:22.346928 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:57:22.349102 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:57:22.349315 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.349570 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:57:22.349738 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.388468 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.130:33123
I20260616 19:57:22.388556 7564 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.130:33123 every 8 connection(s)
I20260616 19:57:22.393311 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:22.400669 7568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:22.402449 7569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:22.405781 7571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:22.406442 7565 heartbeater.cc:344] Connected to a master server at 127.6.159.190:37049
I20260616 19:57:22.406922 7565 heartbeater.cc:461] Registering TS with master...
I20260616 19:57:22.407960 7565 heartbeater.cc:507] Master 127.6.159.190:37049 requested a full tablet report, sending...
I20260616 19:57:22.408607 6782 server_base.cc:1061] running on GCE node
I20260616 19:57:22.409544 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:22.409808 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:22.410020 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639842409988 us; error 0 us; skew 500 ppm
I20260616 19:57:22.410319 7363 ts_manager.cc:194] Registered new tserver with Master: cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:22.410598 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:22.411773 7363 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37596
I20260616 19:57:22.413218 6782 webserver.cc:533] Webserver started at http://127.6.159.131:35431/ using document root <none> and password file <none>
I20260616 19:57:22.413726 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:22.413944 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:22.414206 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:22.415337 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-2-root/instance:
uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
format_stamp: "Formatted at 2026-06-16 19:57:22 on dist-test-slave-1c0m"
I20260616 19:57:22.419656 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260616 19:57:22.422973 7576 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.423931 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260616 19:57:22.424261 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-2-root
uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
format_stamp: "Formatted at 2026-06-16 19:57:22 on dist-test-slave-1c0m"
I20260616 19:57:22.424535 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1781639809409380-6782-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:22.452612 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:22.453817 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:22.455222 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:57:22.457511 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:57:22.457734 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.457998 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:57:22.458170 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:22.509583 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.131:46165
I20260616 19:57:22.509681 7638 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.131:46165 every 8 connection(s)
I20260616 19:57:22.523370 7639 heartbeater.cc:344] Connected to a master server at 127.6.159.190:37049
I20260616 19:57:22.523736 7639 heartbeater.cc:461] Registering TS with master...
I20260616 19:57:22.524540 7639 heartbeater.cc:507] Master 127.6.159.190:37049 requested a full tablet report, sending...
I20260616 19:57:22.526610 7363 ts_manager.cc:194] Registered new tserver with Master: 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:22.527443 6782 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014488492s
I20260616 19:57:22.527938 7363 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37606
I20260616 19:57:22.551479 7363 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37608:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260616 19:57:22.554428 7363 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260616 19:57:22.637122 7530 tablet_service.cc:1511] Processing CreateTablet for tablet 35f7a7aa4c0c44bcabadfb9d695bd10a (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260616 19:57:22.638767 7530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f7a7aa4c0c44bcabadfb9d695bd10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.641263 7529 tablet_service.cc:1511] Processing CreateTablet for tablet a0a2433db6d7427eb36d9fd0bfc007e7 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260616 19:57:22.642689 7529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a2433db6d7427eb36d9fd0bfc007e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.645282 7528 tablet_service.cc:1511] Processing CreateTablet for tablet 3b5597a4844942afa3ab6c15d724daa4 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260616 19:57:22.646140 7527 tablet_service.cc:1511] Processing CreateTablet for tablet b544f2459569420ead4107f46e3e55a9 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260616 19:57:22.646644 7528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5597a4844942afa3ab6c15d724daa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.647387 7527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b544f2459569420ead4107f46e3e55a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.651021 7526 tablet_service.cc:1511] Processing CreateTablet for tablet 3896cf9554884244a3b2f77ba8685dd3 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260616 19:57:22.652453 7526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3896cf9554884244a3b2f77ba8685dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.656687 7525 tablet_service.cc:1511] Processing CreateTablet for tablet 4cbddbd283a44495bca6efbd40fd23e8 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260616 19:57:22.658061 7525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cbddbd283a44495bca6efbd40fd23e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.661908 7524 tablet_service.cc:1511] Processing CreateTablet for tablet 9e02544881fd41b183d3fc4d90498a3b (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260616 19:57:22.663341 7524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e02544881fd41b183d3fc4d90498a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.669450 7654 tablet_bootstrap.cc:492] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:22.669361 7523 tablet_service.cc:1511] Processing CreateTablet for tablet 2f1a7f3e16714967a0fb3b27146ea01d (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260616 19:57:22.671985 7523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1a7f3e16714967a0fb3b27146ea01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.679729 7528 tablet_service.cc:1511] Processing CreateTablet for tablet 4017d59f530e4dfd9ddfe034b3b85233 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260616 19:57:22.681205 7528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4017d59f530e4dfd9ddfe034b3b85233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.686789 7602 tablet_service.cc:1511] Processing CreateTablet for tablet 3b5597a4844942afa3ab6c15d724daa4 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260616 19:57:22.689101 7604 tablet_service.cc:1511] Processing CreateTablet for tablet 35f7a7aa4c0c44bcabadfb9d695bd10a (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260616 19:57:22.689263 7601 tablet_service.cc:1511] Processing CreateTablet for tablet b544f2459569420ead4107f46e3e55a9 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260616 19:57:22.691419 7603 tablet_service.cc:1511] Processing CreateTablet for tablet a0a2433db6d7427eb36d9fd0bfc007e7 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260616 19:57:22.693363 7596 tablet_service.cc:1511] Processing CreateTablet for tablet 4017d59f530e4dfd9ddfe034b3b85233 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260616 19:57:22.694535 7599 tablet_service.cc:1511] Processing CreateTablet for tablet 4cbddbd283a44495bca6efbd40fd23e8 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260616 19:57:22.696472 7597 tablet_service.cc:1511] Processing CreateTablet for tablet 2f1a7f3e16714967a0fb3b27146ea01d (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260616 19:57:22.698695 7595 tablet_service.cc:1511] Processing CreateTablet for tablet 3c553b3058af48baad4bdf3a5943a8ed (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260616 19:57:22.701344 7453 tablet_service.cc:1511] Processing CreateTablet for tablet 3b5597a4844942afa3ab6c15d724daa4 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260616 19:57:22.702687 7453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5597a4844942afa3ab6c15d724daa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.703070 7524 tablet_service.cc:1511] Processing CreateTablet for tablet 3c553b3058af48baad4bdf3a5943a8ed (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260616 19:57:22.705384 7452 tablet_service.cc:1511] Processing CreateTablet for tablet b544f2459569420ead4107f46e3e55a9 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260616 19:57:22.706692 7452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b544f2459569420ead4107f46e3e55a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.706900 7524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c553b3058af48baad4bdf3a5943a8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.709364 7451 tablet_service.cc:1511] Processing CreateTablet for tablet 3896cf9554884244a3b2f77ba8685dd3 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260616 19:57:22.710676 7451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3896cf9554884244a3b2f77ba8685dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.712759 7449 tablet_service.cc:1511] Processing CreateTablet for tablet 9e02544881fd41b183d3fc4d90498a3b (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260616 19:57:22.714123 7449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e02544881fd41b183d3fc4d90498a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.715677 7654 tablet_bootstrap.cc:654] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.717446 7450 tablet_service.cc:1511] Processing CreateTablet for tablet 4cbddbd283a44495bca6efbd40fd23e8 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260616 19:57:22.718850 7450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cbddbd283a44495bca6efbd40fd23e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.693360 7600 tablet_service.cc:1511] Processing CreateTablet for tablet 3896cf9554884244a3b2f77ba8685dd3 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260616 19:57:22.698697 7455 tablet_service.cc:1511] Processing CreateTablet for tablet 35f7a7aa4c0c44bcabadfb9d695bd10a (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260616 19:57:22.725592 7654 tablet_bootstrap.cc:492] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:22.699852 7454 tablet_service.cc:1511] Processing CreateTablet for tablet a0a2433db6d7427eb36d9fd0bfc007e7 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260616 19:57:22.726163 7654 ts_tablet_manager.cc:1403] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.057s user 0.013s sys 0.003s
I20260616 19:57:22.726253 7455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f7a7aa4c0c44bcabadfb9d695bd10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.726994 7454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a2433db6d7427eb36d9fd0bfc007e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.729233 7654 raft_consensus.cc:359] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.730000 7654 raft_consensus.cc:385] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.730281 7654 raft_consensus.cc:740] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.731179 7654 consensus_queue.cc:260] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.691529 7598 tablet_service.cc:1511] Processing CreateTablet for tablet 9e02544881fd41b183d3fc4d90498a3b (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260616 19:57:22.733173 7602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5597a4844942afa3ab6c15d724daa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.733783 7599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cbddbd283a44495bca6efbd40fd23e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.736445 7603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a2433db6d7427eb36d9fd0bfc007e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.742488 7448 tablet_service.cc:1511] Processing CreateTablet for tablet 2f1a7f3e16714967a0fb3b27146ea01d (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260616 19:57:22.743851 7448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1a7f3e16714967a0fb3b27146ea01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.744406 7596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4017d59f530e4dfd9ddfe034b3b85233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.745064 7604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f7a7aa4c0c44bcabadfb9d695bd10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.748482 7595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c553b3058af48baad4bdf3a5943a8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.750373 7597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1a7f3e16714967a0fb3b27146ea01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.751791 7447 tablet_service.cc:1511] Processing CreateTablet for tablet 4017d59f530e4dfd9ddfe034b3b85233 (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260616 19:57:22.753145 7447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4017d59f530e4dfd9ddfe034b3b85233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.753914 7601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b544f2459569420ead4107f46e3e55a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.757426 7600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3896cf9554884244a3b2f77ba8685dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.757861 7446 tablet_service.cc:1511] Processing CreateTablet for tablet 3c553b3058af48baad4bdf3a5943a8ed (DEFAULT_TABLE table=test-workload [id=84dee79edb4e41b4b3cfe4a813373672]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260616 19:57:22.758934 7598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e02544881fd41b183d3fc4d90498a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.759537 7446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c553b3058af48baad4bdf3a5943a8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:22.842804 7654 ts_tablet_manager.cc:1434] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.116s user 0.017s sys 0.020s
I20260616 19:57:22.845649 7565 heartbeater.cc:499] Master 127.6.159.190:37049 was elected leader, sending a full tablet report...
I20260616 19:57:22.846519 7654 tablet_bootstrap.cc:492] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:22.858153 7658 tablet_bootstrap.cc:492] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:22.858922 7657 tablet_bootstrap.cc:492] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:22.863329 7657 tablet_bootstrap.cc:654] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.865109 7658 tablet_bootstrap.cc:654] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.876284 7658 tablet_bootstrap.cc:492] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:22.876807 7658 ts_tablet_manager.cc:1403] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.004s
I20260616 19:57:22.878422 7654 tablet_bootstrap.cc:654] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.878978 7657 tablet_bootstrap.cc:492] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:22.879027 7658 raft_consensus.cc:359] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.879666 7658 raft_consensus.cc:385] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.879709 7657 ts_tablet_manager.cc:1403] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.009s
I20260616 19:57:22.879896 7658 raft_consensus.cc:740] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.880744 7658 consensus_queue.cc:260] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.883050 7490 heartbeater.cc:499] Master 127.6.159.190:37049 was elected leader, sending a full tablet report...
I20260616 19:57:22.883179 7657 raft_consensus.cc:359] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.883922 7654 tablet_bootstrap.cc:492] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:22.883949 7657 raft_consensus.cc:385] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.884517 7657 raft_consensus.cc:740] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.884634 7654 ts_tablet_manager.cc:1403] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.038s user 0.013s sys 0.000s
I20260616 19:57:22.884673 7658 ts_tablet_manager.cc:1434] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260616 19:57:22.885457 7657 consensus_queue.cc:260] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.886030 7658 tablet_bootstrap.cc:492] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:22.887705 7654 raft_consensus.cc:359] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.888497 7654 raft_consensus.cc:385] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.888809 7654 raft_consensus.cc:740] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.889665 7654 consensus_queue.cc:260] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
W20260616 19:57:22.892758 7566 tablet.cc:2406] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260616 19:57:22.893478 7654 ts_tablet_manager.cc:1434] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260616 19:57:22.894498 7654 tablet_bootstrap.cc:492] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:22.895712 7658 tablet_bootstrap.cc:654] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.901139 7654 tablet_bootstrap.cc:654] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.915246 7654 tablet_bootstrap.cc:492] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:22.915767 7654 ts_tablet_manager.cc:1403] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.008s
I20260616 19:57:22.918625 7654 raft_consensus.cc:359] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.919332 7654 raft_consensus.cc:385] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.919629 7654 raft_consensus.cc:740] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.920574 7654 consensus_queue.cc:260] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.924993 7654 ts_tablet_manager.cc:1434] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260616 19:57:22.925938 7654 tablet_bootstrap.cc:492] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:22.932140 7654 tablet_bootstrap.cc:654] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.939549 7639 heartbeater.cc:499] Master 127.6.159.190:37049 was elected leader, sending a full tablet report...
I20260616 19:57:22.940289 7658 tablet_bootstrap.cc:492] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:22.940441 7657 ts_tablet_manager.cc:1434] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.060s user 0.028s sys 0.024s
I20260616 19:57:22.940897 7658 ts_tablet_manager.cc:1403] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.055s user 0.025s sys 0.028s
I20260616 19:57:22.941450 7657 tablet_bootstrap.cc:492] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:22.943928 7658 raft_consensus.cc:359] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.944844 7658 raft_consensus.cc:385] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.945250 7658 raft_consensus.cc:740] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.946199 7658 consensus_queue.cc:260] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.948074 7657 tablet_bootstrap.cc:654] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.948789 7658 ts_tablet_manager.cc:1434] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260616 19:57:22.949900 7658 tablet_bootstrap.cc:492] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:22.958199 7658 tablet_bootstrap.cc:654] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.975327 7657 tablet_bootstrap.cc:492] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:22.975474 7654 tablet_bootstrap.cc:492] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:22.976007 7654 ts_tablet_manager.cc:1403] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.050s user 0.024s sys 0.023s
I20260616 19:57:22.976014 7657 ts_tablet_manager.cc:1403] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.035s user 0.013s sys 0.019s
I20260616 19:57:22.978760 7654 raft_consensus.cc:359] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.979578 7654 raft_consensus.cc:385] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.979895 7654 raft_consensus.cc:740] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.980049 7657 raft_consensus.cc:359] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.981022 7657 raft_consensus.cc:385] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.980903 7654 consensus_queue.cc:260] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.981397 7657 raft_consensus.cc:740] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.982329 7657 consensus_queue.cc:260] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:22.983340 7658 tablet_bootstrap.cc:492] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:22.983844 7658 ts_tablet_manager.cc:1403] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.007s
I20260616 19:57:22.984766 7657 ts_tablet_manager.cc:1434] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260616 19:57:22.985627 7654 ts_tablet_manager.cc:1434] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260616 19:57:22.985724 7657 tablet_bootstrap.cc:492] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:22.986686 7654 tablet_bootstrap.cc:492] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:22.987174 7658 raft_consensus.cc:359] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:22.987996 7658 raft_consensus.cc:385] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:22.988375 7658 raft_consensus.cc:740] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:22.989331 7658 consensus_queue.cc:260] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:22.991994 7658 ts_tablet_manager.cc:1434] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260616 19:57:22.992643 7657 tablet_bootstrap.cc:654] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:22.993114 7658 tablet_bootstrap.cc:492] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:22.993206 7654 tablet_bootstrap.cc:654] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.000850 7656 raft_consensus.cc:493] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.001762 7658 tablet_bootstrap.cc:654] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.001403 7656 raft_consensus.cc:515] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.004127 7657 tablet_bootstrap.cc:492] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.004221 7656 leader_election.cc:290] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.004720 7657 ts_tablet_manager.cc:1403] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.007s
I20260616 19:57:23.007913 7657 raft_consensus.cc:359] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.008890 7657 raft_consensus.cc:385] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.009264 7657 raft_consensus.cc:740] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.010533 7657 consensus_queue.cc:260] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.013644 7657 ts_tablet_manager.cc:1434] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260616 19:57:23.014642 7657 tablet_bootstrap.cc:492] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:23.022710 7657 tablet_bootstrap.cc:654] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.037038 7657 tablet_bootstrap.cc:492] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.037685 7657 ts_tablet_manager.cc:1403] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.008s
I20260616 19:57:23.037557 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5597a4844942afa3ab6c15d724daa4" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
W20260616 19:57:23.039331 7505 leader_election.cc:343] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.041627 7657 raft_consensus.cc:359] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:23.042608 7657 raft_consensus.cc:385] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.042991 7657 raft_consensus.cc:740] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.044404 7657 consensus_queue.cc:260] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:23.050097 7657 ts_tablet_manager.cc:1434] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.012s user 0.007s sys 0.002s
I20260616 19:57:23.051425 7657 tablet_bootstrap.cc:492] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:23.059705 7657 tablet_bootstrap.cc:654] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.068111 7656 raft_consensus.cc:493] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.068670 7656 raft_consensus.cc:515] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.069239 7654 tablet_bootstrap.cc:492] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:23.069613 7657 tablet_bootstrap.cc:492] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.069706 7654 ts_tablet_manager.cc:1403] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.083s user 0.029s sys 0.021s
I20260616 19:57:23.070262 7657 ts_tablet_manager.cc:1403] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.004s
I20260616 19:57:23.070839 7656 leader_election.cc:290] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.071820 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5597a4844942afa3ab6c15d724daa4" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.072891 7658 tablet_bootstrap.cc:492] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:23.073450 7658 ts_tablet_manager.cc:1403] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.081s user 0.027s sys 0.044s
I20260616 19:57:23.073601 7657 raft_consensus.cc:359] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.074090 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.074535 7657 raft_consensus.cc:385] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260616 19:57:23.074618 7506 leader_election.cc:343] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.074960 7657 raft_consensus.cc:740] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.075114 7506 leader_election.cc:304] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258
W20260616 19:57:23.075495 7505 leader_election.cc:343] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.076076 7656 raft_consensus.cc:2749] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:23.076336 7658 raft_consensus.cc:359] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.076105 7657 consensus_queue.cc:260] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.076983 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.077069 7658 raft_consensus.cc:385] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.072939 7654 raft_consensus.cc:359] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.077649 7654 raft_consensus.cc:385] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.077966 7654 raft_consensus.cc:740] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
W20260616 19:57:23.078212 7506 leader_election.cc:343] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.078631 7506 leader_election.cc:304] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:23.078670 7654 consensus_queue.cc:260] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.079268 7656 raft_consensus.cc:2749] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:23.079924 7658 raft_consensus.cc:740] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.080842 7657 ts_tablet_manager.cc:1434] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260616 19:57:23.080735 7658 consensus_queue.cc:260] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.082242 7657 tablet_bootstrap.cc:492] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:23.082973 7658 ts_tablet_manager.cc:1434] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260616 19:57:23.082930 7656 raft_consensus.cc:493] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.084113 7658 tablet_bootstrap.cc:492] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:23.084303 7656 raft_consensus.cc:515] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.087359 7656 leader_election.cc:290] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.089437 7657 tablet_bootstrap.cc:654] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:23.093490 7411 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:23.099583 7658 tablet_bootstrap.cc:654] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.099814 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.100575 7465 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 0.
I20260616 19:57:23.101912 7505 leader_election.cc:304] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.102197 7657 tablet_bootstrap.cc:492] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.102773 7657 ts_tablet_manager.cc:1403] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.005s
I20260616 19:57:23.103084 7676 raft_consensus.cc:2804] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.103529 7676 raft_consensus.cc:493] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.103806 7654 ts_tablet_manager.cc:1434] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.034s user 0.012s sys 0.002s
I20260616 19:57:23.104094 7676 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.104892 7654 tablet_bootstrap.cc:492] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:23.105952 7657 raft_consensus.cc:359] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.106698 7657 raft_consensus.cc:385] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.107026 7657 raft_consensus.cc:740] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.107853 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.107821 7657 consensus_queue.cc:260] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
W20260616 19:57:23.109566 7506 leader_election.cc:343] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.110442 7657 ts_tablet_manager.cc:1434] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260616 19:57:23.111410 7657 tablet_bootstrap.cc:492] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:23.111718 7654 tablet_bootstrap.cc:654] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.112783 7676 raft_consensus.cc:515] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.116366 7676 leader_election.cc:290] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.117657 7657 tablet_bootstrap.cc:654] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.117604 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:23.118283 7465 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.124339 7658 tablet_bootstrap.cc:492] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:23.125073 7658 ts_tablet_manager.cc:1403] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.041s user 0.011s sys 0.006s
I20260616 19:57:23.129473 7658 raft_consensus.cc:359] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.130580 7658 raft_consensus.cc:385] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.131134 7658 raft_consensus.cc:740] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.132303 7658 consensus_queue.cc:260] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.135401 7658 ts_tablet_manager.cc:1434] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.010s user 0.004s sys 0.003s
I20260616 19:57:23.136656 7658 tablet_bootstrap.cc:492] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:23.138751 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.145705 7658 tablet_bootstrap.cc:654] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.151707 7662 raft_consensus.cc:493] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.152359 7662 raft_consensus.cc:515] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:23.153391 7676 raft_consensus.cc:493] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.154026 7676 raft_consensus.cc:515] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.157481 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.158339 7464 raft_consensus.cc:2468] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 0.
I20260616 19:57:23.156569 7676 leader_election.cc:290] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
W20260616 19:57:23.159806 7506 leader_election.cc:343] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.159823 7505 leader_election.cc:304] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.161230 7676 raft_consensus.cc:2804] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.161724 7676 raft_consensus.cc:493] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.162091 7676 raft_consensus.cc:3060] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.162643 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.163484 7614 raft_consensus.cc:2468] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 0.
I20260616 19:57:23.167375 7662 leader_election.cc:290] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.169661 7465 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 1.
I20260616 19:57:23.171550 7676 raft_consensus.cc:515] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.174551 7657 tablet_bootstrap.cc:492] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.174887 7657 ts_tablet_manager.cc:1403] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.064s user 0.025s sys 0.036s
I20260616 19:57:23.175096 7658 tablet_bootstrap.cc:492] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:23.175715 7658 ts_tablet_manager.cc:1403] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.039s user 0.019s sys 0.005s
I20260616 19:57:23.177481 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.177567 7657 raft_consensus.cc:359] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.178121 7614 raft_consensus.cc:3060] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.178290 7657 raft_consensus.cc:385] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.178625 7657 raft_consensus.cc:740] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.179481 7657 consensus_queue.cc:260] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.180094 7658 raft_consensus.cc:359] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.181187 7658 raft_consensus.cc:385] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.181568 7658 raft_consensus.cc:740] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.182492 7658 consensus_queue.cc:260] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.185065 7658 ts_tablet_manager.cc:1434] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260616 19:57:23.186578 7658 tablet_bootstrap.cc:492] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:23.187119 7657 ts_tablet_manager.cc:1434] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.012s user 0.004s sys 0.004s
I20260616 19:57:23.185075 7614 raft_consensus.cc:2468] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 1.
I20260616 19:57:23.185163 7676 leader_election.cc:290] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.188077 7657 tablet_bootstrap.cc:492] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:23.189986 7506 leader_election.cc:304] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.191924 7654 tablet_bootstrap.cc:492] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:23.192431 7654 ts_tablet_manager.cc:1403] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.088s user 0.010s sys 0.036s
I20260616 19:57:23.194574 7658 tablet_bootstrap.cc:654] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.195632 7676 raft_consensus.cc:2804] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.195333 7654 raft_consensus.cc:359] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.196846 7676 raft_consensus.cc:697] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:23.197093 7657 tablet_bootstrap.cc:654] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.196087 7654 raft_consensus.cc:385] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.197762 7676 consensus_queue.cc:237] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.199652 7654 raft_consensus.cc:740] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.201581 7505 leader_election.cc:304] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 2ef425cb67d74a2fb7695789c4bd5aae
I20260616 19:57:23.201545 7654 consensus_queue.cc:260] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.202692 7656 raft_consensus.cc:2804] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.203156 7656 raft_consensus.cc:697] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:23.204319 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:23.204126 7656 consensus_queue.cc:237] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.204816 7654 ts_tablet_manager.cc:1434] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.012s user 0.005s sys 0.002s
I20260616 19:57:23.204823 7465 raft_consensus.cc:3060] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.205868 7654 tablet_bootstrap.cc:492] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:23.214392 7658 tablet_bootstrap.cc:492] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:23.214959 7658 ts_tablet_manager.cc:1403] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.004s
I20260616 19:57:23.215117 7465 raft_consensus.cc:2468] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 1.
I20260616 19:57:23.218146 7654 tablet_bootstrap.cc:654] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.217990 7658 raft_consensus.cc:359] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.218991 7658 raft_consensus.cc:385] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.219352 7658 raft_consensus.cc:740] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.220172 7658 consensus_queue.cc:260] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.222523 7658 ts_tablet_manager.cc:1434] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260616 19:57:23.224624 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
I20260616 19:57:23.225131 7663 raft_consensus.cc:493] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.225557 7663 raft_consensus.cc:515] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
W20260616 19:57:23.226931 7430 leader_election.cc:343] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.228608 7663 leader_election.cc:290] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255)
I20260616 19:57:23.236120 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.236831 7614 raft_consensus.cc:2468] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 0.
I20260616 19:57:23.237448 7654 tablet_bootstrap.cc:492] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:23.238062 7654 ts_tablet_manager.cc:1403] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.004s
I20260616 19:57:23.238240 7431 leader_election.cc:304] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters: cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:23.239112 7662 raft_consensus.cc:2804] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.239511 7662 raft_consensus.cc:493] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.239897 7662 raft_consensus.cc:3060] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.243057 7654 raft_consensus.cc:359] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:23.243865 7654 raft_consensus.cc:385] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.244273 7654 raft_consensus.cc:740] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.243762 7692 raft_consensus.cc:493] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.245153 7654 consensus_queue.cc:260] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:23.245546 7692 raft_consensus.cc:515] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.247952 7654 ts_tablet_manager.cc:1434] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20260616 19:57:23.248934 7654 tablet_bootstrap.cc:492] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:23.249044 7657 tablet_bootstrap.cc:492] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.249542 7657 ts_tablet_manager.cc:1403] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.062s user 0.021s sys 0.012s
I20260616 19:57:23.250348 7363 catalog_manager.cc:5696] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.252306 7657 raft_consensus.cc:359] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.254808 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
I20260616 19:57:23.255345 7654 tablet_bootstrap.cc:654] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.255453 7657 raft_consensus.cc:385] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.256354 7657 raft_consensus.cc:740] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.255671 7540 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 0.
I20260616 19:57:23.257014 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.257620 7614 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 0.
I20260616 19:57:23.257710 7430 leader_election.cc:304] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.259853 7696 raft_consensus.cc:493] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.251993 7662 raft_consensus.cc:515] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:23.257303 7657 consensus_queue.cc:260] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.262670 7657 ts_tablet_manager.cc:1434] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260616 19:57:23.263294 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.266482 7540 raft_consensus.cc:3060] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.272786 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.273307 7657 tablet_bootstrap.cc:492] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:23.254895 7692 leader_election.cc:290] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.273363 7614 raft_consensus.cc:3060] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.276928 7692 raft_consensus.cc:2804] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.277534 7692 raft_consensus.cc:493] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.278048 7692 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.285192 7614 raft_consensus.cc:2468] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
I20260616 19:57:23.260505 7696 raft_consensus.cc:515] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.286715 7431 leader_election.cc:304] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:23.264209 7662 leader_election.cc:290] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.287616 7662 raft_consensus.cc:2804] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.290872 7539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5597a4844942afa3ab6c15d724daa4" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
I20260616 19:57:23.224545 7658 tablet_bootstrap.cc:492] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:23.291847 7539 raft_consensus.cc:2468] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 0.
I20260616 19:57:23.293521 7430 leader_election.cc:304] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.296666 7540 raft_consensus.cc:2468] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
I20260616 19:57:23.299415 7658 tablet_bootstrap.cc:654] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.302579 7657 tablet_bootstrap.cc:654] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.306190 7693 raft_consensus.cc:2804] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.306711 7693 raft_consensus.cc:493] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.307139 7693 raft_consensus.cc:3060] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.308003 7662 raft_consensus.cc:697] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:23.309013 7662 consensus_queue.cc:237] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:23.308737 7360 catalog_manager.cc:5696] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.314182 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5597a4844942afa3ab6c15d724daa4" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.314538 7696 leader_election.cc:290] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.314983 7614 raft_consensus.cc:2468] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 0.
I20260616 19:57:23.317209 7656 raft_consensus.cc:493] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.317703 7656 raft_consensus.cc:515] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.318315 7654 tablet_bootstrap.cc:492] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:23.318833 7654 ts_tablet_manager.cc:1403] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.070s user 0.026s sys 0.024s
I20260616 19:57:23.319837 7656 leader_election.cc:290] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.321277 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.321691 7654 raft_consensus.cc:359] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.322517 7654 raft_consensus.cc:385] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.322847 7654 raft_consensus.cc:740] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
W20260616 19:57:23.323573 7505 leader_election.cc:343] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.327405 7654 consensus_queue.cc:260] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.330087 7654 ts_tablet_manager.cc:1434] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260616 19:57:23.331198 7654 tablet_bootstrap.cc:492] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:23.318318 7692 raft_consensus.cc:515] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.327039 7693 raft_consensus.cc:515] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.335554 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.341527 7540 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.343101 7654 tablet_bootstrap.cc:654] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.343859 7538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5597a4844942afa3ab6c15d724daa4" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.344554 7538 raft_consensus.cc:3060] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.346740 7693 leader_election.cc:290] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.347546 7612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5597a4844942afa3ab6c15d724daa4" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.335892 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.348212 7612 raft_consensus.cc:3060] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.348446 7614 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.351400 7693 raft_consensus.cc:493] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.336463 7692 leader_election.cc:290] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.336691 7663 raft_consensus.cc:493] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.352303 7538 raft_consensus.cc:2468] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
I20260616 19:57:23.351966 7693 raft_consensus.cc:515] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.352664 7663 raft_consensus.cc:515] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.353543 7430 leader_election.cc:304] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.337437 7613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.355139 7613 raft_consensus.cc:2468] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 0.
I20260616 19:57:23.338775 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.355929 7537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
I20260616 19:57:23.355994 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.356251 7612 raft_consensus.cc:2468] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
I20260616 19:57:23.340728 7539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
I20260616 19:57:23.356755 7537 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 0.
I20260616 19:57:23.356837 7611 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 0.
I20260616 19:57:23.357142 7539 raft_consensus.cc:2393] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ef425cb67d74a2fb7695789c4bd5aae in current term 1: Already voted for candidate 9f84cb1d073a4fd79a5309423cb71258 in this term.
I20260616 19:57:23.358141 7430 leader_election.cc:304] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.358304 7693 leader_election.cc:290] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.354825 7663 leader_election.cc:290] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:23.354809 7692 raft_consensus.cc:2804] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.359411 7693 raft_consensus.cc:2804] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.359795 7692 raft_consensus.cc:697] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:23.359972 7693 raft_consensus.cc:493] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.360533 7693 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.360803 7692 consensus_queue.cc:237] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.361239 7539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
I20260616 19:57:23.362282 7658 tablet_bootstrap.cc:492] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:23.362730 7658 ts_tablet_manager.cc:1403] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.138s user 0.005s sys 0.012s
I20260616 19:57:23.363220 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.363865 7465 raft_consensus.cc:2468] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 0.
I20260616 19:57:23.365407 7506 leader_election.cc:304] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:23.366341 7579 leader_election.cc:304] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:23.366549 7656 raft_consensus.cc:2804] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.366979 7656 raft_consensus.cc:493] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.367326 7656 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.367422 7579 leader_election.cc:304] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae; no voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:23.368369 7663 raft_consensus.cc:2804] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.368291 7693 raft_consensus.cc:515] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.368898 7663 raft_consensus.cc:493] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.369395 7663 raft_consensus.cc:3060] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.372665 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.373230 7611 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.375394 7656 raft_consensus.cc:515] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.365376 7658 raft_consensus.cc:359] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.378697 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:23.375885 7654 tablet_bootstrap.cc:492] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:23.379081 7658 raft_consensus.cc:385] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.379418 7658 raft_consensus.cc:740] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.379452 7654 ts_tablet_manager.cc:1403] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.048s user 0.006s sys 0.007s
I20260616 19:57:23.380977 7656 leader_election.cc:290] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.381307 7540 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
I20260616 19:57:23.381664 7658 consensus_queue.cc:260] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.382354 7430 leader_election.cc:304] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.383996 7611 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
I20260616 19:57:23.384853 7693 leader_election.cc:290] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.385680 7538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.386474 7538 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.382671 7654 raft_consensus.cc:359] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.382503 7465 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.387465 7431 leader_election.cc:304] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:23.388794 7696 raft_consensus.cc:2804] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.388545 7663 raft_consensus.cc:515] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.389277 7696 raft_consensus.cc:697] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:23.390105 7696 consensus_queue.cc:237] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.390540 7663 leader_election.cc:290] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:23.382759 7614 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
W20260616 19:57:23.392764 7579 leader_election.cc:343] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.394104 7654 raft_consensus.cc:385] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.394502 7654 raft_consensus.cc:740] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.393996 7538 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 1.
I20260616 19:57:23.394873 7700 raft_consensus.cc:2804] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.394023 7465 raft_consensus.cc:2468] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 1.
I20260616 19:57:23.397419 7657 tablet_bootstrap.cc:492] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.397339 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:23.397936 7657 ts_tablet_manager.cc:1403] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.125s user 0.012s sys 0.008s
I20260616 19:57:23.398012 7464 raft_consensus.cc:3060] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.386440 7707 raft_consensus.cc:2749] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:23.394330 7363 catalog_manager.cc:5696] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:23.401952 7657 raft_consensus.cc:359] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.402714 7657 raft_consensus.cc:385] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.403003 7657 raft_consensus.cc:740] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.403846 7657 consensus_queue.cc:260] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.404191 7505 leader_election.cc:304] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.406075 7657 ts_tablet_manager.cc:1434] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260616 19:57:23.406903 7657 tablet_bootstrap.cc:492] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae: Bootstrap starting.
I20260616 19:57:23.395506 7654 consensus_queue.cc:260] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.394807 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.396108 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.408100 7614 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:23.408727 7579 leader_election.cc:343] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.409606 7654 ts_tablet_manager.cc:1434] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.030s user 0.003s sys 0.003s
I20260616 19:57:23.395471 7700 raft_consensus.cc:697] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:23.411410 7700 consensus_queue.cc:237] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.412904 7654 tablet_bootstrap.cc:492] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4: Bootstrap starting.
I20260616 19:57:23.413525 7657 tablet_bootstrap.cc:654] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.415797 7656 raft_consensus.cc:2804] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.400125 7658 ts_tablet_manager.cc:1434] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.037s user 0.005s sys 0.002s
I20260616 19:57:23.418522 7658 tablet_bootstrap.cc:492] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:23.416453 7656 raft_consensus.cc:697] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:23.421350 7656 consensus_queue.cc:237] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.422705 7464 raft_consensus.cc:2468] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 1.
I20260616 19:57:23.422917 7614 raft_consensus.cc:2468] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 1.
I20260616 19:57:23.423808 7579 leader_election.cc:304] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters: cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:23.425717 7663 raft_consensus.cc:2804] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.426108 7707 raft_consensus.cc:493] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.426596 7707 raft_consensus.cc:515] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.431774 7707 leader_election.cc:290] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:23.437604 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c553b3058af48baad4bdf3a5943a8ed" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.440469 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c553b3058af48baad4bdf3a5943a8ed" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
W20260616 19:57:23.440840 7579 leader_election.cc:343] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.441313 7540 raft_consensus.cc:2468] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 0.
I20260616 19:57:23.442768 7579 leader_election.cc:304] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:23.444336 7707 raft_consensus.cc:2804] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.445081 7707 raft_consensus.cc:493] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.445946 7707 raft_consensus.cc:3060] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.450204 7658 tablet_bootstrap.cc:654] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.451747 7657 tablet_bootstrap.cc:492] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae: No bootstrap required, opened a new log
I20260616 19:57:23.452239 7657 ts_tablet_manager.cc:1403] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent bootstrapping tablet: real 0.045s user 0.017s sys 0.022s
I20260616 19:57:23.455075 7657 raft_consensus.cc:359] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.455838 7657 raft_consensus.cc:385] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.456182 7657 raft_consensus.cc:740] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.457010 7657 consensus_queue.cc:260] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.458957 7657 ts_tablet_manager.cc:1434] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260616 19:57:23.461555 7654 tablet_bootstrap.cc:654] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.460793 7663 raft_consensus.cc:697] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Becoming Leader. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:23.460031 7361 catalog_manager.cc:5696] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.472704 7707 raft_consensus.cc:515] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.475760 7654 tablet_bootstrap.cc:492] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4: No bootstrap required, opened a new log
I20260616 19:57:23.476320 7654 ts_tablet_manager.cc:1403] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent bootstrapping tablet: real 0.064s user 0.005s sys 0.009s
I20260616 19:57:23.476398 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c553b3058af48baad4bdf3a5943a8ed" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:23.477198 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c553b3058af48baad4bdf3a5943a8ed" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.477764 7540 raft_consensus.cc:3060] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:23.477932 7579 leader_election.cc:343] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:23.479053 7654 raft_consensus.cc:359] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.479732 7654 raft_consensus.cc:385] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.473740 7663 consensus_queue.cc:237] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.480423 7363 catalog_manager.cc:5696] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.479984 7654 raft_consensus.cc:740] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.482426 7363 catalog_manager.cc:5696] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.483500 7654 consensus_queue.cc:260] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.484695 7363 catalog_manager.cc:5696] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.485666 7707 leader_election.cc:290] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:23.485262 7540 raft_consensus.cc:2468] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 1.
I20260616 19:57:23.487516 7579 leader_election.cc:304] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:23.488095 7654 ts_tablet_manager.cc:1434] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260616 19:57:23.488875 7707 raft_consensus.cc:2804] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.491263 7707 raft_consensus.cc:697] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Becoming Leader. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:23.492295 7707 consensus_queue.cc:237] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.490636 7658 tablet_bootstrap.cc:492] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:23.494976 7658 ts_tablet_manager.cc:1403] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.078s user 0.014s sys 0.005s
I20260616 19:57:23.497537 7658 raft_consensus.cc:359] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.498554 7658 raft_consensus.cc:385] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.498978 7658 raft_consensus.cc:740] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.499456 7361 catalog_manager.cc:5696] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131). New cstate: current_term: 1 leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:23.499815 7658 consensus_queue.cc:260] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.502596 7658 ts_tablet_manager.cc:1434] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260616 19:57:23.503559 7658 tablet_bootstrap.cc:492] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258: Bootstrap starting.
I20260616 19:57:23.509300 7658 tablet_bootstrap.cc:654] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:23.516829 7658 tablet_bootstrap.cc:492] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258: No bootstrap required, opened a new log
I20260616 19:57:23.517230 7658 ts_tablet_manager.cc:1403] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260616 19:57:23.517846 7361 catalog_manager.cc:5696] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131). New cstate: current_term: 1 leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:23.519614 7658 raft_consensus.cc:359] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.520344 7658 raft_consensus.cc:385] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:23.520612 7658 raft_consensus.cc:740] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Initialized, Role: FOLLOWER
I20260616 19:57:23.521255 7658 consensus_queue.cc:260] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.522899 7658 ts_tablet_manager.cc:1434] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260616 19:57:23.560885 7707 raft_consensus.cc:493] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.561381 7707 raft_consensus.cc:515] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.563194 7707 leader_election.cc:290] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:23.564057 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.564302 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" is_pre_election: true
I20260616 19:57:23.564747 7464 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 0.
I20260616 19:57:23.564865 7540 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 0.
I20260616 19:57:23.565761 7579 leader_election.cc:304] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:23.566390 7707 raft_consensus.cc:2804] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:23.566725 7707 raft_consensus.cc:493] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:23.567056 7707 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.571760 7707 raft_consensus.cc:515] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.573344 7707 leader_election.cc:290] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:23.574177 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:23.574384 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.574661 7464 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.574975 7540 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.579303 7464 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 1.
I20260616 19:57:23.579586 7540 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 1.
I20260616 19:57:23.580343 7579 leader_election.cc:304] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:23.581148 7707 raft_consensus.cc:2804] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:23.581619 7707 raft_consensus.cc:697] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Becoming Leader. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:23.582397 7707 consensus_queue.cc:237] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.589129 7360 catalog_manager.cc:5696] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131). New cstate: current_term: 1 leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:23.627617 7656 consensus_queue.cc:1048] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.638494 7656 consensus_queue.cc:1048] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.696662 7656 raft_consensus.cc:493] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:23.697057 7656 raft_consensus.cc:515] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.698616 7656 leader_election.cc:290] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.699460 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f84cb1d073a4fd79a5309423cb71258" is_pre_election: true
I20260616 19:57:23.699738 7614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" is_pre_election: true
I20260616 19:57:23.700340 7464 raft_consensus.cc:2393] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in current term 1: Already voted for candidate 2ef425cb67d74a2fb7695789c4bd5aae in this term.
I20260616 19:57:23.701272 7505 leader_election.cc:304] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:23.701869 7656 raft_consensus.cc:3060] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.706564 7656 raft_consensus.cc:2749] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:23.726122 7700 consensus_queue.cc:1048] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.736868 7696 consensus_queue.cc:1048] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.754896 7696 consensus_queue.cc:1048] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.761445 7540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
mode: GRACEFUL
new_leader_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:44462
I20260616 19:57:23.762219 7540 raft_consensus.cc:606] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Received request to transfer leadership to TS 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:23.766323 7692 consensus_queue.cc:1048] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.776391 7540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
mode: GRACEFUL
new_leader_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:44462
I20260616 19:57:23.777035 7540 raft_consensus.cc:606] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Received request to transfer leadership to TS 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:23.791967 7614 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.795025 7611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
mode: GRACEFUL
new_leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:56530
I20260616 19:57:23.795629 7611 raft_consensus.cc:606] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Received request to transfer leadership to TS cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:23.798831 7656 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.800563 7656 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.806959 7676 raft_consensus.cc:993] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4: : Instructing follower 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:23.807425 7676 raft_consensus.cc:1081] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Signalling peer 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:23.809154 7464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:58936
I20260616 19:57:23.809916 7464 raft_consensus.cc:493] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:23.810359 7464 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:23.817847 7463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
mode: GRACEFUL
new_leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:58950
I20260616 19:57:23.818447 7463 raft_consensus.cc:606] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Received request to transfer leadership to TS cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:23.820683 7464 raft_consensus.cc:515] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.821779 7465 raft_consensus.cc:1240] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Rejecting Update request from peer cc4339c8cc364cefbb7ec9fb2d748ae4 for earlier term 1. Current term is 2. Ops: []
I20260616 19:57:23.826015 7611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
mode: GRACEFUL
new_leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:56530
I20260616 19:57:23.826649 7611 raft_consensus.cc:606] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Received request to transfer leadership to TS cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:23.827605 7656 consensus_queue.cc:1059] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.827997 7538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:23.828272 7464 leader_election.cc:290] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 2 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.828575 7538 raft_consensus.cc:3055] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:23.829069 7538 raft_consensus.cc:740] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:23.830056 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.829878 7538 consensus_queue.cc:260] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.830976 7538 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:23.836956 7676 consensus_queue.cc:1048] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.840611 7538 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 2.
W20260616 19:57:23.841178 7656 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260616 19:57:23.841796 7430 leader_election.cc:304] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 2ef425cb67d74a2fb7695789c4bd5aae
I20260616 19:57:23.842980 7692 raft_consensus.cc:2804] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:23.843139 7700 consensus_queue.cc:1048] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.843516 7692 raft_consensus.cc:697] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:23.844568 7692 consensus_queue.cc:237] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.849916 7700 consensus_queue.cc:1048] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.853649 7360 catalog_manager.cc:5696] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 1 to 2, leader changed from cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130) to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.859292 7693 consensus_queue.cc:1048] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.861920 7693 consensus_queue.cc:1048] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.880307 7692 raft_consensus.cc:993] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258: : Instructing follower cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:23.880781 7692 raft_consensus.cc:1081] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Signalling peer cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:23.882776 7538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:44438
I20260616 19:57:23.883325 7538 raft_consensus.cc:493] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:23.883726 7538 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:23.883085 7677 consensus_queue.cc:1048] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:23.890172 7538 raft_consensus.cc:515] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.894274 7538 leader_election.cc:290] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:23.895422 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:23.896005 7611 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:23.897084 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:23.897881 7465 raft_consensus.cc:3055] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:23.898219 7465 raft_consensus.cc:740] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:23.899155 7465 consensus_queue.cc:260] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.902820 7611 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 2.
I20260616 19:57:23.902832 7465 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:23.906579 7506 leader_election.cc:304] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:23.907389 7676 raft_consensus.cc:2804] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:23.907902 7676 raft_consensus.cc:697] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:23.908713 7676 consensus_queue.cc:237] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:23.911300 7465 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 2.
I20260616 19:57:23.915448 7361 catalog_manager.cc:5696] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 1 to 2, leader changed from 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129) to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:23.957600 7465 raft_consensus.cc:3060] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:23.963522 7707 consensus_queue.cc:1048] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:23.975533 7707 consensus_queue.cc:1048] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:24.028970 7676 raft_consensus.cc:993] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4: : Instructing follower 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:24.029160 7707 consensus_queue.cc:1048] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:24.029533 7677 raft_consensus.cc:1081] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Signalling peer 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:24.030988 7464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:58936
I20260616 19:57:24.031562 7464 raft_consensus.cc:493] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:24.031937 7464 raft_consensus.cc:3060] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.041141 7464 raft_consensus.cc:515] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.041710 7711 consensus_queue.cc:1048] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:24.043890 7538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:24.044493 7538 raft_consensus.cc:3055] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:24.044831 7538 raft_consensus.cc:740] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:24.045153 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a2433db6d7427eb36d9fd0bfc007e7" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:24.045717 7611 raft_consensus.cc:3060] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.045611 7538 consensus_queue.cc:260] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.046793 7538 raft_consensus.cc:3060] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.049320 7464 leader_election.cc:290] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 2 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:24.053079 7538 raft_consensus.cc:2468] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 2.
I20260616 19:57:24.054399 7430 leader_election.cc:304] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:24.055254 7693 raft_consensus.cc:2804] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:24.055492 7611 raft_consensus.cc:2468] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 2.
I20260616 19:57:24.055685 7693 raft_consensus.cc:697] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:24.056559 7693 consensus_queue.cc:237] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.058127 7711 raft_consensus.cc:993] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae: : Instructing follower cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:24.058553 7663 raft_consensus.cc:1081] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Signalling peer cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:24.060913 7540 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:44448
I20260616 19:57:24.061513 7540 raft_consensus.cc:493] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:24.061959 7540 raft_consensus.cc:3060] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.063416 7360 catalog_manager.cc:5696] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 1 to 2, leader changed from cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130) to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:24.065003 7663 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:24.071441 7540 raft_consensus.cc:515] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.074131 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:24.074486 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4017d59f530e4dfd9ddfe034b3b85233" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:24.074713 7464 raft_consensus.cc:3060] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.075054 7611 raft_consensus.cc:3055] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:24.075394 7611 raft_consensus.cc:740] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:24.076560 7540 leader_election.cc:290] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:24.076232 7611 consensus_queue.cc:260] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.077387 7611 raft_consensus.cc:3060] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.080118 7464 raft_consensus.cc:2468] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 2.
I20260616 19:57:24.081310 7505 leader_election.cc:304] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:24.081503 7707 raft_consensus.cc:993] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae: : Instructing follower cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:24.082046 7707 raft_consensus.cc:1081] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Signalling peer cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:24.082408 7677 raft_consensus.cc:2804] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:24.082592 7611 raft_consensus.cc:2468] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 2.
I20260616 19:57:24.082897 7677 raft_consensus.cc:697] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:24.084319 7677 consensus_queue.cc:237] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.085594 7538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:44448
I20260616 19:57:24.087641 7538 raft_consensus.cc:493] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:24.088011 7538 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.091758 7711 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:24.093220 7360 catalog_manager.cc:5696] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 1 to 2, leader changed from 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131) to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:24.094700 7538 raft_consensus.cc:515] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.100276 7538 leader_election.cc:290] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:24.100188 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:24.112553 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:24.113260 7611 raft_consensus.cc:3055] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:24.113603 7611 raft_consensus.cc:740] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:24.114427 7611 consensus_queue.cc:260] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.115687 7611 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.120754 7611 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 2.
I20260616 19:57:24.121701 7506 leader_election.cc:304] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters: 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:24.122359 7677 raft_consensus.cc:2804] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:24.122792 7677 raft_consensus.cc:697] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:24.123505 7677 consensus_queue.cc:237] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:24.129637 7360 catalog_manager.cc:5696] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 1 to 2, leader changed from 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131) to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:24.224546 7538 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.225744 7693 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.233246 7611 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.239880 7611 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.241180 7662 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.368647 7464 raft_consensus.cc:1275] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.369827 7677 consensus_queue.cc:1048] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.376873 7611 raft_consensus.cc:1275] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.378173 7656 consensus_queue.cc:1048] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.450606 7611 raft_consensus.cc:1275] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.451862 7693 consensus_queue.cc:1048] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.459230 7538 raft_consensus.cc:1275] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.460781 7700 consensus_queue.cc:1048] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.484193 7464 raft_consensus.cc:1275] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.485415 7677 consensus_queue.cc:1048] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.492895 7611 raft_consensus.cc:1275] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.494589 7656 consensus_queue.cc:1048] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.628508 7611 raft_consensus.cc:1275] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.629724 7464 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:24.629729 7656 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:24.636452 7464 raft_consensus.cc:1275] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:24.637975 7677 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:26.866151 6782 auto_leader_rebalancer-test.cc:365] The leader distribution is
I20260616 19:57:26.866815 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:26.883406 7464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
mode: GRACEFUL
new_leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:58964
I20260616 19:57:26.884001 7464 raft_consensus.cc:606] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Received request to transfer leadership to TS 2ef425cb67d74a2fb7695789c4bd5aae
I20260616 19:57:26.886529 7464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
mode: GRACEFUL
new_leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:58964
I20260616 19:57:26.887025 7464 raft_consensus.cc:606] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Received request to transfer leadership to TS 2ef425cb67d74a2fb7695789c4bd5aae
I20260616 19:57:26.888087 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260616 19:57:26.888502 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:27.285336 7745 raft_consensus.cc:993] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: : Instructing follower 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:27.285882 7749 raft_consensus.cc:1081] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Signalling peer 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:27.287495 7611 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:56518
I20260616 19:57:27.287976 7611 raft_consensus.cc:493] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:27.288324 7611 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:27.293272 7611 raft_consensus.cc:515] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:27.294905 7611 leader_election.cc:290] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 3 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:27.295806 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:27.295974 7538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:27.296490 7464 raft_consensus.cc:3055] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:27.296491 7538 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:27.296976 7464 raft_consensus.cc:740] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:27.297785 7464 consensus_queue.cc:260] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:27.298971 7464 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:27.303550 7538 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 3.
I20260616 19:57:27.305050 7579 leader_election.cc:304] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:27.305555 7464 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 3.
I20260616 19:57:27.305769 7711 raft_consensus.cc:2804] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:27.306958 7711 raft_consensus.cc:697] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 LEADER]: Becoming Leader. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:27.307804 7711 consensus_queue.cc:237] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:27.314412 7361 catalog_manager.cc:5696] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae reported cstate change: term changed from 2 to 3, leader changed from 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129) to 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131). New cstate: current_term: 3 leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:27.362860 7745 raft_consensus.cc:993] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258: : Instructing follower 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:27.363253 7749 raft_consensus.cc:1081] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Signalling peer 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:27.364746 7611 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:56518
I20260616 19:57:27.365239 7611 raft_consensus.cc:493] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:27.365528 7611 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:27.369865 7611 raft_consensus.cc:515] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:27.371416 7611 leader_election.cc:290] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 2 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:27.372267 7464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:27.372517 7538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:27.372824 7464 raft_consensus.cc:3055] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:27.373083 7538 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:27.373126 7464 raft_consensus.cc:740] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:27.373891 7464 consensus_queue.cc:260] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:27.374754 7464 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:27.377811 7538 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 2.
I20260616 19:57:27.378793 7579 leader_election.cc:304] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:27.379355 7464 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 2.
I20260616 19:57:27.379666 7711 raft_consensus.cc:2804] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:27.380264 7711 raft_consensus.cc:697] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 LEADER]: Becoming Leader. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:27.381081 7711 consensus_queue.cc:237] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:27.389430 7361 catalog_manager.cc:5696] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae reported cstate change: term changed from 1 to 2, leader changed from 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129) to 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131). New cstate: current_term: 2 leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:27.730996 7538 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:27.732331 7711 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:27.742234 7465 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:27.744132 7760 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:57:27.773245 7465 raft_consensus.cc:1275] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:27.774542 7711 consensus_queue.cc:1048] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:27.782878 7538 raft_consensus.cc:1275] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:27.784430 7750 consensus_queue.cc:1048] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:27.889376 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:27.896566 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:27.897091 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:28.898066 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:28.903399 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:28.903859 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:29.904986 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:29.911038 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:29.911433 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:30.912468 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:30.918586 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:30.918915 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:31.919706 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:31.924942 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:31.925350 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:32.926347 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:32.931838 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:32.932343 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:33.933274 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:33.938453 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:33.938882 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:34.939800 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:34.945026 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:34.945461 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:35.946487 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:35.951627 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:35.951973 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:36.982525 6782 auto_leader_rebalancer-test.cc:384] The leader distribution is
I20260616 19:57:36.993286 7611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
mode: GRACEFUL
new_leader_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:56530
I20260616 19:57:36.993897 7611 raft_consensus.cc:606] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 LEADER]: Received request to transfer leadership to TS 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:36.998276 7611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
mode: GRACEFUL
new_leader_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:56530
I20260616 19:57:36.998715 7611 raft_consensus.cc:606] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 LEADER]: Received request to transfer leadership to TS 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:37.003185 7540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
mode: GRACEFUL
new_leader_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:44462
I20260616 19:57:37.003654 7540 raft_consensus.cc:606] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Received request to transfer leadership to TS 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:37.008134 7540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
mode: GRACEFUL
new_leader_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:44462
I20260616 19:57:37.008630 7540 raft_consensus.cc:606] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Received request to transfer leadership to TS 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:37.014076 7540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
mode: GRACEFUL
new_leader_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:44462
I20260616 19:57:37.014526 7540 raft_consensus.cc:606] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Received request to transfer leadership to TS 9f84cb1d073a4fd79a5309423cb71258
I20260616 19:57:37.075790 7746 raft_consensus.cc:993] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4: : Instructing follower 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.076391 7777 raft_consensus.cc:1081] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Signalling peer 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.077991 7465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:58936
I20260616 19:57:37.078476 7465 raft_consensus.cc:493] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:37.078755 7465 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:37.083956 7465 raft_consensus.cc:515] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.085690 7465 leader_election.cc:290] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 2 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:37.086581 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:37.086846 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:37.087316 7611 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:37.087302 7540 raft_consensus.cc:3055] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:37.087761 7540 raft_consensus.cc:740] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:37.088671 7540 consensus_queue.cc:260] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.089957 7540 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:37.094543 7611 raft_consensus.cc:2468] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 2.
I20260616 19:57:37.095906 7431 leader_election.cc:304] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:37.096812 7772 raft_consensus.cc:2804] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:37.096823 7540 raft_consensus.cc:2468] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 2.
I20260616 19:57:37.098104 7772 raft_consensus.cc:697] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:37.099159 7772 consensus_queue.cc:237] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.106827 7363 catalog_manager.cc:5696] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 1 to 2, leader changed from cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130) to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:37.138191 7746 raft_consensus.cc:993] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4: : Instructing follower 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.138574 7777 raft_consensus.cc:1081] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Signalling peer 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.139889 7465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:58936
I20260616 19:57:37.140363 7465 raft_consensus.cc:493] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:37.140672 7465 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.145437 7465 raft_consensus.cc:515] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.147220 7465 leader_election.cc:290] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 3 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:37.148008 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:37.148144 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1a7f3e16714967a0fb3b27146ea01d" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:37.148568 7540 raft_consensus.cc:3055] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:37.148703 7611 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.148865 7540 raft_consensus.cc:740] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:37.149628 7540 consensus_queue.cc:260] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.150499 7540 raft_consensus.cc:3060] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.153126 7611 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 3.
I20260616 19:57:37.154095 7431 leader_election.cc:304] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:37.154851 7772 raft_consensus.cc:2804] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:37.154973 7540 raft_consensus.cc:2468] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 3.
I20260616 19:57:37.155325 7772 raft_consensus.cc:697] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:37.156193 7772 consensus_queue.cc:237] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.162896 7363 catalog_manager.cc:5696] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 2 to 3, leader changed from cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130) to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:37.311247 7746 raft_consensus.cc:993] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4: : Instructing follower 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.311676 7746 raft_consensus.cc:1081] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Signalling peer 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.313042 7465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:58936
I20260616 19:57:37.313498 7465 raft_consensus.cc:493] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:37.313809 7465 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.317947 7465 raft_consensus.cc:515] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.319343 7465 leader_election.cc:290] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 3 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:37.320284 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:37.320312 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:37.320955 7540 raft_consensus.cc:3055] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:37.320995 7611 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.321280 7540 raft_consensus.cc:740] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:37.321951 7540 consensus_queue.cc:260] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.322814 7540 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.325567 7611 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 3.
I20260616 19:57:37.326475 7431 leader_election.cc:304] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, 9f84cb1d073a4fd79a5309423cb71258; no voters:
I20260616 19:57:37.327207 7772 raft_consensus.cc:2804] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:37.327404 7540 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 3.
I20260616 19:57:37.327673 7772 raft_consensus.cc:697] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:37.328997 7772 consensus_queue.cc:237] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.336205 7363 catalog_manager.cc:5696] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 2 to 3, leader changed from cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130) to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:37.361996 7775 raft_consensus.cc:993] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae: : Instructing follower 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.362740 7786 raft_consensus.cc:1081] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 LEADER]: Signalling peer 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.364486 7465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:58944
I20260616 19:57:37.365028 7465 raft_consensus.cc:493] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:37.365343 7465 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.369675 7465 raft_consensus.cc:515] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.370893 7465 leader_election.cc:290] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 3 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:37.371708 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:37.371838 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b544f2459569420ead4107f46e3e55a9" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:37.372325 7540 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.372372 7611 raft_consensus.cc:3055] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:37.372761 7611 raft_consensus.cc:740] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:37.373421 7611 consensus_queue.cc:260] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.374266 7611 raft_consensus.cc:3060] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:37.376740 7540 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 3.
I20260616 19:57:37.377676 7430 leader_election.cc:304] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:37.378300 7611 raft_consensus.cc:2468] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 3.
I20260616 19:57:37.378335 7772 raft_consensus.cc:2804] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:37.378876 7772 raft_consensus.cc:697] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:37.379755 7772 consensus_queue.cc:237] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.386865 7363 catalog_manager.cc:5696] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 2 to 3, leader changed from 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131) to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:37.450874 7775 raft_consensus.cc:993] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae: : Instructing follower 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.451397 7786 raft_consensus.cc:1081] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 LEADER]: Signalling peer 9f84cb1d073a4fd79a5309423cb71258 to start an election
I20260616 19:57:37.452766 7465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
from {username='slave'} at 127.0.0.1:58944
I20260616 19:57:37.453200 7465 raft_consensus.cc:493] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:37.453423 7465 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:37.457448 7465 raft_consensus.cc:515] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.458937 7465 leader_election.cc:290] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 4 election: Requested vote from peers cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:37.459710 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:37.459857 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "9f84cb1d073a4fd79a5309423cb71258" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:37.460316 7540 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:37.460323 7611 raft_consensus.cc:3055] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:57:37.460896 7611 raft_consensus.cc:740] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:37.461673 7611 consensus_queue.cc:260] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.462515 7611 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:37.464959 7540 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 4.
I20260616 19:57:37.466195 7430 leader_election.cc:304] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f84cb1d073a4fd79a5309423cb71258, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:37.467146 7772 raft_consensus.cc:2804] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:57:37.467607 7772 raft_consensus.cc:697] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 LEADER]: Becoming Leader. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:37.468047 7611 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f84cb1d073a4fd79a5309423cb71258 in term 4.
I20260616 19:57:37.468451 7772 consensus_queue.cc:237] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:37.475893 7363 catalog_manager.cc:5696] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 reported cstate change: term changed from 3 to 4, leader changed from 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131) to 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129). New cstate: current_term: 4 leader_uuid: "9f84cb1d073a4fd79a5309423cb71258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:37.572959 7611 raft_consensus.cc:1275] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:37.574080 7772 consensus_queue.cc:1048] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:37.583935 7540 raft_consensus.cc:1275] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:37.585564 7778 consensus_queue.cc:1048] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:57:37.633534 7611 raft_consensus.cc:1275] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:37.634691 7778 consensus_queue.cc:1048] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:37.643798 7538 raft_consensus.cc:1275] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:37.645309 7791 consensus_queue.cc:1048] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:57:37.742954 7614 raft_consensus.cc:1275] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:37.744113 7791 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:37.752657 7538 raft_consensus.cc:1275] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:37.754359 7772 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:37.763015 7614 raft_consensus.cc:1275] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:37.764351 7772 consensus_queue.cc:1048] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:37.773264 7538 raft_consensus.cc:1275] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:37.774895 7778 consensus_queue.cc:1048] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:37.908382 7538 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:37.909468 7772 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:37.918615 7614 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 FOLLOWER]: Refusing update from remote peer 9f84cb1d073a4fd79a5309423cb71258: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:37.920269 7772 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:40.052560 6782 auto_leader_rebalancer-test.cc:399] The leader distribution is
I20260616 19:57:40.053333 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:40.061126 7465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
mode: GRACEFUL
new_leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:58964
I20260616 19:57:40.061949 7465 raft_consensus.cc:606] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Received request to transfer leadership to TS cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:40.064561 7465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
mode: GRACEFUL
new_leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:58964
I20260616 19:57:40.064975 7465 raft_consensus.cc:606] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 LEADER]: Received request to transfer leadership to TS cc4339c8cc364cefbb7ec9fb2d748ae4
I20260616 19:57:40.067090 7465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
mode: GRACEFUL
new_leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:58964
I20260616 19:57:40.067505 7465 raft_consensus.cc:606] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Received request to transfer leadership to TS 2ef425cb67d74a2fb7695789c4bd5aae
I20260616 19:57:40.069697 7465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b"
dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
mode: GRACEFUL
new_leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:58964
I20260616 19:57:40.070109 7465 raft_consensus.cc:606] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Received request to transfer leadership to TS 2ef425cb67d74a2fb7695789c4bd5aae
I20260616 19:57:40.071053 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260616 19:57:40.071553 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:40.141311 7800 raft_consensus.cc:993] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258: : Instructing follower 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:40.141733 7800 raft_consensus.cc:1081] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Signalling peer 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:40.143298 7611 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:56518
I20260616 19:57:40.143748 7611 raft_consensus.cc:493] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:40.144007 7611 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:40.149492 7611 raft_consensus.cc:515] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.151358 7611 leader_election.cc:290] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 3 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:40.152469 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:40.153306 7465 raft_consensus.cc:3055] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:57:40.153074 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbddbd283a44495bca6efbd40fd23e8" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:40.153604 7465 raft_consensus.cc:740] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:40.153867 7540 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:40.154633 7465 consensus_queue.cc:260] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.156201 7465 raft_consensus.cc:3060] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:57:40.160673 7540 raft_consensus.cc:2468] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 3.
I20260616 19:57:40.162120 7579 leader_election.cc:304] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:40.162199 7465 raft_consensus.cc:2468] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 3.
I20260616 19:57:40.163275 7786 raft_consensus.cc:2804] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:57:40.164337 7786 raft_consensus.cc:697] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 LEADER]: Becoming Leader. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:40.165282 7786 consensus_queue.cc:237] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.172425 7361 catalog_manager.cc:5696] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae reported cstate change: term changed from 2 to 3, leader changed from 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129) to 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131). New cstate: current_term: 3 leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:40.194582 7805 raft_consensus.cc:993] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: : Instructing follower cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:40.195019 7800 raft_consensus.cc:1081] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 LEADER]: Signalling peer cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:40.196477 7540 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:44438
I20260616 19:57:40.196909 7540 raft_consensus.cc:493] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:40.197168 7540 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:57:40.201555 7540 raft_consensus.cc:515] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.203344 7540 leader_election.cc:290] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 5 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165)
I20260616 19:57:40.204176 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:40.204368 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3896cf9554884244a3b2f77ba8685dd3" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:40.204739 7465 raft_consensus.cc:3055] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:57:40.204793 7611 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:57:40.205085 7465 raft_consensus.cc:740] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:40.205844 7465 consensus_queue.cc:260] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.206763 7465 raft_consensus.cc:3060] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:57:40.210435 7611 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 5.
I20260616 19:57:40.211516 7506 leader_election.cc:304] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:40.212028 7465 raft_consensus.cc:2468] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 5.
I20260616 19:57:40.212282 7746 raft_consensus.cc:2804] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:57:40.213610 7746 raft_consensus.cc:697] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 5 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:40.214299 7746 consensus_queue.cc:237] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.220520 7361 catalog_manager.cc:5696] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 4 to 5, leader changed from 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129) to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 5 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:40.288882 7805 raft_consensus.cc:993] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258: : Instructing follower 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:40.289383 7800 raft_consensus.cc:1081] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Signalling peer 2ef425cb67d74a2fb7695789c4bd5aae to start an election
I20260616 19:57:40.290917 7611 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b"
dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
from {username='slave'} at 127.0.0.1:56518
I20260616 19:57:40.291391 7611 raft_consensus.cc:493] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:40.291796 7611 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:40.296070 7611 raft_consensus.cc:515] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.297477 7611 leader_election.cc:290] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 4 election: Requested vote from peers 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255), cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123)
I20260616 19:57:40.298288 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:40.298592 7540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e02544881fd41b183d3fc4d90498a3b" candidate_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
I20260616 19:57:40.298858 7465 raft_consensus.cc:3055] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:57:40.299096 7540 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:40.299155 7465 raft_consensus.cc:740] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:40.300050 7465 consensus_queue.cc:260] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.301206 7465 raft_consensus.cc:3060] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:57:40.305650 7540 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 4.
I20260616 19:57:40.306691 7579 leader_election.cc:304] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:40.307214 7465 raft_consensus.cc:2468] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef425cb67d74a2fb7695789c4bd5aae in term 4.
I20260616 19:57:40.307350 7786 raft_consensus.cc:2804] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:57:40.307825 7786 raft_consensus.cc:697] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 LEADER]: Becoming Leader. State: Replica: 2ef425cb67d74a2fb7695789c4bd5aae, State: Running, Role: LEADER
I20260616 19:57:40.308691 7786 consensus_queue.cc:237] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } }
I20260616 19:57:40.315822 7361 catalog_manager.cc:5696] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae reported cstate change: term changed from 3 to 4, leader changed from 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129) to 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131). New cstate: current_term: 4 leader_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:40.406361 7805 raft_consensus.cc:993] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258: : Instructing follower cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:40.406816 7805 raft_consensus.cc:1081] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Signalling peer cc4339c8cc364cefbb7ec9fb2d748ae4 to start an election
I20260616 19:57:40.408227 7540 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a"
dest_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4"
from {username='slave'} at 127.0.0.1:44438
I20260616 19:57:40.408703 7540 raft_consensus.cc:493] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:57:40.408931 7540 raft_consensus.cc:3060] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:40.413003 7540 raft_consensus.cc:515] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:40.414361 7540 leader_election.cc:290] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Requested vote from peers 2ef425cb67d74a2fb7695789c4bd5aae (127.6.159.131:46165), 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255)
I20260616 19:57:40.415360 7465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f84cb1d073a4fd79a5309423cb71258"
I20260616 19:57:40.415422 7611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f7a7aa4c0c44bcabadfb9d695bd10a" candidate_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ef425cb67d74a2fb7695789c4bd5aae"
I20260616 19:57:40.415867 7465 raft_consensus.cc:3055] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:57:40.415959 7611 raft_consensus.cc:3060] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:40.416146 7465 raft_consensus.cc:740] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f84cb1d073a4fd79a5309423cb71258, State: Running, Role: LEADER
I20260616 19:57:40.416926 7465 consensus_queue.cc:260] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:40.417764 7465 raft_consensus.cc:3060] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:40.420451 7611 raft_consensus.cc:2468] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 2.
I20260616 19:57:40.421377 7506 leader_election.cc:304] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef425cb67d74a2fb7695789c4bd5aae, cc4339c8cc364cefbb7ec9fb2d748ae4; no voters:
I20260616 19:57:40.422045 7746 raft_consensus.cc:2804] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:40.422044 7465 raft_consensus.cc:2468] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4339c8cc364cefbb7ec9fb2d748ae4 in term 2.
I20260616 19:57:40.422675 7746 raft_consensus.cc:697] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Becoming Leader. State: Replica: cc4339c8cc364cefbb7ec9fb2d748ae4, State: Running, Role: LEADER
I20260616 19:57:40.423455 7746 consensus_queue.cc:237] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } }
I20260616 19:57:40.430285 7361 catalog_manager.cc:5696] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 reported cstate change: term changed from 1 to 2, leader changed from 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129) to cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:40.563390 7465 raft_consensus.cc:1275] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:40.564524 7786 consensus_queue.cc:1048] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:40.575484 7540 raft_consensus.cc:1275] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:57:40.576972 7786 consensus_queue.cc:1048] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:57:40.668017 7465 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 5 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:57:40.669139 7746 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:57:40.677650 7611 raft_consensus.cc:1275] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 5 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:57:40.679505 7824 consensus_queue.cc:1048] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:57:40.777309 7465 raft_consensus.cc:1275] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 4 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:40.778388 7786 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:57:40.787101 7540 raft_consensus.cc:1275] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Refusing update from remote peer 2ef425cb67d74a2fb7695789c4bd5aae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:57:40.788556 7820 consensus_queue.cc:1048] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4339c8cc364cefbb7ec9fb2d748ae4" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:57:40.975847 7611 raft_consensus.cc:1275] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:40.977165 7746 consensus_queue.cc:1048] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef425cb67d74a2fb7695789c4bd5aae" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:40.987459 7465 raft_consensus.cc:1275] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Refusing update from remote peer cc4339c8cc364cefbb7ec9fb2d748ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:57:40.988745 7824 consensus_queue.cc:1048] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f84cb1d073a4fd79a5309423cb71258" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:57:41.072535 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:41.078004 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:41.078382 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:42.079289 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:42.085731 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:42.086257 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:43.087289 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:43.092756 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:43.093207 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:44.094125 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:44.100829 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:44.101269 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:45.102147 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:45.108645 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:45.109169 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:46.110047 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:46.115118 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:46.115465 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:47.116325 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:47.122541 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:47.123049 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:48.124091 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:48.130812 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:48.131302 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:49.132299 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:57:49.137626 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260616 19:57:49.138028 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:57:50.168326 6782 auto_leader_rebalancer-test.cc:410] The leader distribution is
I20260616 19:57:50.168947 6782 tablet_server.cc:179] TabletServer@127.6.159.129:0 shutting down...
W20260616 19:57:50.180373 7579 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260616 19:57:50.186412 7579 consensus_peers.cc:597] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae -> Peer 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255): Couldn't send request to peer 9f84cb1d073a4fd79a5309423cb71258. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:57:50.204104 6782 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260616 19:57:50.205001 6782 tablet_replica.cc:333] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.205744 6782 raft_consensus.cc:2243] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.206370 6782 raft_consensus.cc:2272] T 4cbddbd283a44495bca6efbd40fd23e8 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.208519 6782 tablet_replica.cc:333] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.209029 6782 raft_consensus.cc:2243] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.209894 6782 raft_consensus.cc:2272] T 2f1a7f3e16714967a0fb3b27146ea01d P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.211763 6782 tablet_replica.cc:333] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.212388 6782 raft_consensus.cc:2243] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.213263 6782 raft_consensus.cc:2272] T a0a2433db6d7427eb36d9fd0bfc007e7 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.215157 6782 tablet_replica.cc:333] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.215669 6782 raft_consensus.cc:2243] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.216064 6782 raft_consensus.cc:2272] T 3c553b3058af48baad4bdf3a5943a8ed P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.217867 6782 tablet_replica.cc:333] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.218379 6782 raft_consensus.cc:2243] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.219182 6782 raft_consensus.cc:2272] T 3b5597a4844942afa3ab6c15d724daa4 P 9f84cb1d073a4fd79a5309423cb71258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.220957 6782 tablet_replica.cc:333] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.221462 6782 raft_consensus.cc:2243] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.221876 6782 raft_consensus.cc:2272] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.223641 6782 tablet_replica.cc:333] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.224200 6782 raft_consensus.cc:2243] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.224679 6782 raft_consensus.cc:2272] T 9e02544881fd41b183d3fc4d90498a3b P 9f84cb1d073a4fd79a5309423cb71258 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.226563 6782 tablet_replica.cc:333] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.227032 6782 raft_consensus.cc:2243] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.227495 6782 raft_consensus.cc:2272] T 4017d59f530e4dfd9ddfe034b3b85233 P 9f84cb1d073a4fd79a5309423cb71258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.229193 6782 tablet_replica.cc:333] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.229650 6782 raft_consensus.cc:2243] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.230404 6782 raft_consensus.cc:2272] T b544f2459569420ead4107f46e3e55a9 P 9f84cb1d073a4fd79a5309423cb71258 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.232131 6782 tablet_replica.cc:333] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258: stopping tablet replica
I20260616 19:57:50.232604 6782 raft_consensus.cc:2243] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.233037 6782 raft_consensus.cc:2272] T 3896cf9554884244a3b2f77ba8685dd3 P 9f84cb1d073a4fd79a5309423cb71258 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.263191 6782 tablet_server.cc:196] TabletServer@127.6.159.129:0 shutdown complete.
I20260616 19:57:50.284006 6782 tablet_server.cc:179] TabletServer@127.6.159.130:0 shutting down...
I20260616 19:57:50.312431 6782 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260616 19:57:50.313305 6782 tablet_replica.cc:333] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.314235 6782 raft_consensus.cc:2243] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.315099 6782 raft_consensus.cc:2272] T 35f7a7aa4c0c44bcabadfb9d695bd10a P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.317102 6782 tablet_replica.cc:333] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.317579 6782 raft_consensus.cc:2243] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 5 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.318303 6782 raft_consensus.cc:2272] T 3896cf9554884244a3b2f77ba8685dd3 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.320011 6782 tablet_replica.cc:333] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.320595 6782 raft_consensus.cc:2243] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.321002 6782 raft_consensus.cc:2272] T a0a2433db6d7427eb36d9fd0bfc007e7 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.322670 6782 tablet_replica.cc:333] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.323101 6782 raft_consensus.cc:2243] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.323437 6782 raft_consensus.cc:2272] T 2f1a7f3e16714967a0fb3b27146ea01d P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.325156 6782 tablet_replica.cc:333] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.325659 6782 raft_consensus.cc:2243] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.326159 6782 raft_consensus.cc:2272] T 4cbddbd283a44495bca6efbd40fd23e8 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.327869 6782 tablet_replica.cc:333] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.328433 6782 raft_consensus.cc:2243] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.329309 6782 raft_consensus.cc:2272] T 4017d59f530e4dfd9ddfe034b3b85233 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.331035 6782 tablet_replica.cc:333] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.331610 6782 raft_consensus.cc:2243] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.332041 6782 raft_consensus.cc:2272] T b544f2459569420ead4107f46e3e55a9 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.333678 6782 tablet_replica.cc:333] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.334157 6782 raft_consensus.cc:2243] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.334589 6782 raft_consensus.cc:2272] T 9e02544881fd41b183d3fc4d90498a3b P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.336278 6782 tablet_replica.cc:333] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.336704 6782 raft_consensus.cc:2243] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.337102 6782 raft_consensus.cc:2272] T 3c553b3058af48baad4bdf3a5943a8ed P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.338725 6782 tablet_replica.cc:333] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4: stopping tablet replica
I20260616 19:57:50.339138 6782 raft_consensus.cc:2243] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.339552 6782 raft_consensus.cc:2272] T 3b5597a4844942afa3ab6c15d724daa4 P cc4339c8cc364cefbb7ec9fb2d748ae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.367317 6782 tablet_server.cc:196] TabletServer@127.6.159.130:0 shutdown complete.
W20260616 19:57:50.383677 7579 consensus_peers.cc:597] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae -> Peer cc4339c8cc364cefbb7ec9fb2d748ae4 (127.6.159.130:33123): Couldn't send request to peer cc4339c8cc364cefbb7ec9fb2d748ae4. Status: Network error: Client connection negotiation failed: client connection to 127.6.159.130:33123: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260616 19:57:50.393671 6782 tablet_server.cc:179] TabletServer@127.6.159.131:0 shutting down...
W20260616 19:57:50.419149 7579 consensus_peers.cc:597] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae -> Peer 9f84cb1d073a4fd79a5309423cb71258 (127.6.159.129:43255): Couldn't send request to peer 9f84cb1d073a4fd79a5309423cb71258. Status: Network error: Client connection negotiation failed: client connection to 127.6.159.129:43255: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260616 19:57:50.424444 6782 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260616 19:57:50.425415 6782 tablet_replica.cc:333] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.426210 6782 raft_consensus.cc:2243] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 5 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.426653 6782 raft_consensus.cc:2272] T 3896cf9554884244a3b2f77ba8685dd3 P 2ef425cb67d74a2fb7695789c4bd5aae [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.428457 6782 tablet_replica.cc:333] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.429013 6782 raft_consensus.cc:2243] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.429380 6782 raft_consensus.cc:2272] T a0a2433db6d7427eb36d9fd0bfc007e7 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.431118 6782 tablet_replica.cc:333] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.431564 6782 raft_consensus.cc:2243] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.431926 6782 raft_consensus.cc:2272] T 35f7a7aa4c0c44bcabadfb9d695bd10a P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.433570 6782 tablet_replica.cc:333] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.433988 6782 raft_consensus.cc:2243] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.434332 6782 raft_consensus.cc:2272] T 3b5597a4844942afa3ab6c15d724daa4 P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.435832 6782 tablet_replica.cc:333] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.436305 6782 raft_consensus.cc:2243] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.437008 6782 raft_consensus.cc:2272] T 3c553b3058af48baad4bdf3a5943a8ed P 2ef425cb67d74a2fb7695789c4bd5aae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.438501 6782 tablet_replica.cc:333] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.438944 6782 raft_consensus.cc:2243] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.439630 6782 raft_consensus.cc:2272] T 9e02544881fd41b183d3fc4d90498a3b P 2ef425cb67d74a2fb7695789c4bd5aae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.441264 6782 tablet_replica.cc:333] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.441682 6782 raft_consensus.cc:2243] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.442085 6782 raft_consensus.cc:2272] T 4017d59f530e4dfd9ddfe034b3b85233 P 2ef425cb67d74a2fb7695789c4bd5aae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.443575 6782 tablet_replica.cc:333] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.443974 6782 raft_consensus.cc:2243] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.444399 6782 raft_consensus.cc:2272] T b544f2459569420ead4107f46e3e55a9 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.445952 6782 tablet_replica.cc:333] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.446328 6782 raft_consensus.cc:2243] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260616 19:57:50.446722 6782 raft_consensus.cc:2272] T 2f1a7f3e16714967a0fb3b27146ea01d P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.448261 6782 tablet_replica.cc:333] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae: stopping tablet replica
I20260616 19:57:50.448642 6782 raft_consensus.cc:2243] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.449363 6782 raft_consensus.cc:2272] T 4cbddbd283a44495bca6efbd40fd23e8 P 2ef425cb67d74a2fb7695789c4bd5aae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.477589 6782 tablet_server.cc:196] TabletServer@127.6.159.131:0 shutdown complete.
I20260616 19:57:50.497253 6782 master.cc:562] Master@127.6.159.190:37049 shutting down...
I20260616 19:57:50.517774 6782 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 1 LEADER]: Raft consensus shutting down.
I20260616 19:57:50.518405 6782 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:57:50.518760 6782 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9539693897dc4bb9982fcca5e1267193: stopping tablet replica
I20260616 19:57:50.538326 6782 master.cc:584] Master@127.6.159.190:37049 shutdown complete.
2ef425cb67d74a2fb7695789c4bd5aae 1
9f84cb1d073a4fd79a5309423cb71258 5
cc4339c8cc364cefbb7ec9fb2d748ae4 4
2ef425cb67d74a2fb7695789c4bd5aae 3
9f84cb1d073a4fd79a5309423cb71258 3
cc4339c8cc364cefbb7ec9fb2d748ae4 4
2ef425cb67d74a2fb7695789c4bd5aae 1
9f84cb1d073a4fd79a5309423cb71258 8
cc4339c8cc364cefbb7ec9fb2d748ae4 1
2ef425cb67d74a2fb7695789c4bd5aae 3
9f84cb1d073a4fd79a5309423cb71258 4
cc4339c8cc364cefbb7ec9fb2d748ae4 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28620 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20260616 19:57:50.581877 6782 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.159.190:44129
I20260616 19:57:50.583021 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:50.589279 7843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:50.589490 7844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:50.589365 7846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:50.590487 6782 server_base.cc:1061] running on GCE node
I20260616 19:57:50.591398 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:50.591576 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:50.591691 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639870591679 us; error 0 us; skew 500 ppm
I20260616 19:57:50.592111 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:50.594596 6782 webserver.cc:533] Webserver started at http://127.6.159.190:44269/ using document root <none> and password file <none>
I20260616 19:57:50.595067 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:50.595228 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:50.595463 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:50.596863 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/master-0-root/instance:
uuid: "9602378bb66e49bcae343c682b95adcb"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.601531 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260616 19:57:50.604836 7851 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.605712 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260616 19:57:50.605963 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/master-0-root
uuid: "9602378bb66e49bcae343c682b95adcb"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.606199 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:50.617328 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:50.618470 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:50.655716 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.190:44129
I20260616 19:57:50.655823 7902 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.190:44129 every 8 connection(s)
I20260616 19:57:50.659691 7903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:50.669225 7903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Bootstrap starting.
I20260616 19:57:50.673810 7903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:50.677771 7903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: No bootstrap required, opened a new log
I20260616 19:57:50.679850 7903 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9602378bb66e49bcae343c682b95adcb" member_type: VOTER }
I20260616 19:57:50.680353 7903 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:50.680546 7903 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9602378bb66e49bcae343c682b95adcb, State: Initialized, Role: FOLLOWER
I20260616 19:57:50.681030 7903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9602378bb66e49bcae343c682b95adcb" member_type: VOTER }
I20260616 19:57:50.681434 7903 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260616 19:57:50.681618 7903 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260616 19:57:50.681818 7903 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:50.685998 7903 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9602378bb66e49bcae343c682b95adcb" member_type: VOTER }
I20260616 19:57:50.686468 7903 leader_election.cc:304] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9602378bb66e49bcae343c682b95adcb; no voters:
I20260616 19:57:50.687523 7903 leader_election.cc:290] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260616 19:57:50.687863 7906 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:50.689253 7906 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [term 1 LEADER]: Becoming Leader. State: Replica: 9602378bb66e49bcae343c682b95adcb, State: Running, Role: LEADER
I20260616 19:57:50.689939 7906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9602378bb66e49bcae343c682b95adcb" member_type: VOTER }
I20260616 19:57:50.690613 7903 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [sys.catalog]: configured and running, proceeding with master startup.
I20260616 19:57:50.695257 7908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9602378bb66e49bcae343c682b95adcb. Latest consensus state: current_term: 1 leader_uuid: "9602378bb66e49bcae343c682b95adcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9602378bb66e49bcae343c682b95adcb" member_type: VOTER } }
I20260616 19:57:50.695988 7908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [sys.catalog]: This master's current role is: LEADER
I20260616 19:57:50.697243 7907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9602378bb66e49bcae343c682b95adcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9602378bb66e49bcae343c682b95adcb" member_type: VOTER } }
I20260616 19:57:50.697822 7907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb [sys.catalog]: This master's current role is: LEADER
I20260616 19:57:50.698495 7915 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
I20260616 19:57:50.706058 7915 catalog_manager.cc:1519] Initializing Kudu cluster ID...
I20260616 19:57:50.707060 6782 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260616 19:57:50.714901 7915 catalog_manager.cc:1382] Generated new cluster ID: 4296dbdfab1643b792bea6dff530b493
I20260616 19:57:50.715184 7915 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
I20260616 19:57:50.751883 7915 catalog_manager.cc:1405] Generated new certificate authority record
I20260616 19:57:50.753192 7915 catalog_manager.cc:1539] Loading token signing keys...
I20260616 19:57:50.764664 7915 catalog_manager.cc:6069] T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Generated new TSK 0
I20260616 19:57:50.765264 7915 catalog_manager.cc:1549] Initializing in-progress tserver states...
I20260616 19:57:50.773140 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:50.778977 7924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:50.780067 7925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:50.781888 6782 server_base.cc:1061] running on GCE node
W20260616 19:57:50.781987 7927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:50.783020 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:50.783221 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:50.783398 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639870783365 us; error 0 us; skew 500 ppm
I20260616 19:57:50.783967 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:50.786183 6782 webserver.cc:533] Webserver started at http://127.6.159.129:45761/ using document root <none> and password file <none>
I20260616 19:57:50.786664 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:50.786850 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:50.787130 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:50.788247 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root/instance:
uuid: "b022cac8cb5e46b7a39a748e81e257a7"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.792821 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260616 19:57:50.796464 7932 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.797233 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260616 19:57:50.797547 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root
uuid: "b022cac8cb5e46b7a39a748e81e257a7"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.797850 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:50.807957 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:50.809353 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:50.810916 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:57:50.813367 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:57:50.813614 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.813922 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:57:50.814136 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.855427 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.129:42531
I20260616 19:57:50.855518 7994 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.129:42531 every 8 connection(s)
I20260616 19:57:50.860643 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:50.869767 8000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:50.871222 7999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:50.874363 6782 server_base.cc:1061] running on GCE node
W20260616 19:57:50.875159 8002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:50.875577 7995 heartbeater.cc:344] Connected to a master server at 127.6.159.190:44129
I20260616 19:57:50.876076 7995 heartbeater.cc:461] Registering TS with master...
I20260616 19:57:50.876423 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:50.876693 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:50.876855 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639870876841 us; error 0 us; skew 500 ppm
I20260616 19:57:50.876906 7995 heartbeater.cc:507] Master 127.6.159.190:44129 requested a full tablet report, sending...
I20260616 19:57:50.877471 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:50.879194 7868 ts_manager.cc:194] Registered new tserver with Master: b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:57:50.880483 6782 webserver.cc:533] Webserver started at http://127.6.159.130:45769/ using document root <none> and password file <none>
I20260616 19:57:50.880977 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:50.881192 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:50.881208 7868 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55728
I20260616 19:57:50.881543 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:50.882915 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root/instance:
uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.887315 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260616 19:57:50.890849 8007 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.891916 6782 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260616 19:57:50.892284 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root
uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.892567 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:50.903863 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:50.905200 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:50.906687 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:57:50.909181 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:57:50.909430 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.909711 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:57:50.909915 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.950925 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.130:43051
I20260616 19:57:50.951016 8069 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.130:43051 every 8 connection(s)
I20260616 19:57:50.956367 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:57:50.963708 8073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:50.964862 8074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:57:50.966634 8076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:57:50.967983 6782 server_base.cc:1061] running on GCE node
I20260616 19:57:50.968381 8070 heartbeater.cc:344] Connected to a master server at 127.6.159.190:44129
I20260616 19:57:50.968767 8070 heartbeater.cc:461] Registering TS with master...
I20260616 19:57:50.969033 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:57:50.969277 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:57:50.969457 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639870969442 us; error 0 us; skew 500 ppm
I20260616 19:57:50.969513 8070 heartbeater.cc:507] Master 127.6.159.190:44129 requested a full tablet report, sending...
I20260616 19:57:50.970072 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:57:50.971840 7868 ts_manager.cc:194] Registered new tserver with Master: de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:50.973002 6782 webserver.cc:533] Webserver started at http://127.6.159.131:35487/ using document root <none> and password file <none>
I20260616 19:57:50.973647 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:57:50.973682 7868 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55740
I20260616 19:57:50.973982 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:57:50.974318 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:57:50.975822 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root/instance:
uuid: "6827184e3cef42c2ada15b2d31afee4b"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.980127 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260616 19:57:50.983407 8081 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:50.984254 6782 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260616 19:57:50.984543 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root
uuid: "6827184e3cef42c2ada15b2d31afee4b"
format_stamp: "Formatted at 2026-06-16 19:57:50 on dist-test-slave-1c0m"
I20260616 19:57:50.984786 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:57:51.006778 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:57:51.008116 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:57:51.009596 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:57:51.011796 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:57:51.011972 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:51.012260 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:57:51.012423 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:57:51.050298 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.131:40887
I20260616 19:57:51.050412 8143 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.131:40887 every 8 connection(s)
I20260616 19:57:51.064373 8144 heartbeater.cc:344] Connected to a master server at 127.6.159.190:44129
I20260616 19:57:51.064754 8144 heartbeater.cc:461] Registering TS with master...
I20260616 19:57:51.065466 8144 heartbeater.cc:507] Master 127.6.159.190:44129 requested a full tablet report, sending...
I20260616 19:57:51.067454 7868 ts_manager.cc:194] Registered new tserver with Master: 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:51.067942 6782 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014619212s
I20260616 19:57:51.068742 7868 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55744
I20260616 19:57:51.095140 7868 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55748:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260616 19:57:51.101527 7868 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260616 19:57:51.311620 7960 tablet_service.cc:1511] Processing CreateTablet for tablet 9f03211770a74604a786608de5062cc3 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260616 19:57:51.313346 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f03211770a74604a786608de5062cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.313269 7957 tablet_service.cc:1511] Processing CreateTablet for tablet d38debbe377040e48b9e248db8babc55 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260616 19:57:51.314688 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38debbe377040e48b9e248db8babc55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.316879 7959 tablet_service.cc:1511] Processing CreateTablet for tablet 001495805f9843f0aac7937a5905dc82 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260616 19:57:51.317308 7956 tablet_service.cc:1511] Processing CreateTablet for tablet 7323d4e3a76f4800925df28a0f090072 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260616 19:57:51.318392 7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 001495805f9843f0aac7937a5905dc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.319533 7955 tablet_service.cc:1511] Processing CreateTablet for tablet 9918060aeb7f44c892f2ade3f2ac6841 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260616 19:57:51.320760 7955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9918060aeb7f44c892f2ade3f2ac6841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.321549 8035 tablet_service.cc:1511] Processing CreateTablet for tablet 9f03211770a74604a786608de5062cc3 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260616 19:57:51.322932 8035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f03211770a74604a786608de5062cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.324554 7956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7323d4e3a76f4800925df28a0f090072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.325469 7958 tablet_service.cc:1511] Processing CreateTablet for tablet f7c5954a3dd147cea894b8e392ab2bd6 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260616 19:57:51.326849 7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7c5954a3dd147cea894b8e392ab2bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.329485 8034 tablet_service.cc:1511] Processing CreateTablet for tablet 001495805f9843f0aac7937a5905dc82 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260616 19:57:51.331103 8034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 001495805f9843f0aac7937a5905dc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.333712 8033 tablet_service.cc:1511] Processing CreateTablet for tablet f7c5954a3dd147cea894b8e392ab2bd6 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260616 19:57:51.335409 8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7c5954a3dd147cea894b8e392ab2bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.337703 8032 tablet_service.cc:1511] Processing CreateTablet for tablet d38debbe377040e48b9e248db8babc55 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260616 19:57:51.339499 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38debbe377040e48b9e248db8babc55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.339558 7953 tablet_service.cc:1511] Processing CreateTablet for tablet 5f64cb36408849c989ecda8d19ec4269 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260616 19:57:51.341005 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f64cb36408849c989ecda8d19ec4269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.341722 8031 tablet_service.cc:1511] Processing CreateTablet for tablet 7323d4e3a76f4800925df28a0f090072 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260616 19:57:51.343417 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7323d4e3a76f4800925df28a0f090072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.345759 7952 tablet_service.cc:1511] Processing CreateTablet for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260616 19:57:51.347095 7952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.347334 8108 tablet_service.cc:1511] Processing CreateTablet for tablet 001495805f9843f0aac7937a5905dc82 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260616 19:57:51.349442 8107 tablet_service.cc:1511] Processing CreateTablet for tablet f7c5954a3dd147cea894b8e392ab2bd6 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260616 19:57:51.349740 7951 tablet_service.cc:1511] Processing CreateTablet for tablet 303be6b16595410aae7147c6026435ff (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260616 19:57:51.376025 8028 tablet_service.cc:1511] Processing CreateTablet for tablet 5f64cb36408849c989ecda8d19ec4269 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260616 19:57:51.377059 8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 001495805f9843f0aac7937a5905dc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.377192 7951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 303be6b16595410aae7147c6026435ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.377398 8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f64cb36408849c989ecda8d19ec4269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.377707 8107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7c5954a3dd147cea894b8e392ab2bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.381085 8027 tablet_service.cc:1511] Processing CreateTablet for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260616 19:57:51.382434 8027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.385394 8026 tablet_service.cc:1511] Processing CreateTablet for tablet 303be6b16595410aae7147c6026435ff (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260616 19:57:51.386787 8026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 303be6b16595410aae7147c6026435ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.345788 8109 tablet_service.cc:1511] Processing CreateTablet for tablet 9f03211770a74604a786608de5062cc3 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260616 19:57:51.389192 8109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f03211770a74604a786608de5062cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.351495 8106 tablet_service.cc:1511] Processing CreateTablet for tablet d38debbe377040e48b9e248db8babc55 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260616 19:57:51.401134 8106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38debbe377040e48b9e248db8babc55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.353801 8105 tablet_service.cc:1511] Processing CreateTablet for tablet 7323d4e3a76f4800925df28a0f090072 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260616 19:57:51.405076 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7323d4e3a76f4800925df28a0f090072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.356074 8104 tablet_service.cc:1511] Processing CreateTablet for tablet 9918060aeb7f44c892f2ade3f2ac6841 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260616 19:57:51.409037 8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9918060aeb7f44c892f2ade3f2ac6841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.358490 8103 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4d4bfa06e04d2d8de1b13263001b1b (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260616 19:57:51.413087 8103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4d4bfa06e04d2d8de1b13263001b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.365134 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 5f64cb36408849c989ecda8d19ec4269 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260616 19:57:51.367781 8101 tablet_service.cc:1511] Processing CreateTablet for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260616 19:57:51.369623 8100 tablet_service.cc:1511] Processing CreateTablet for tablet 303be6b16595410aae7147c6026435ff (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260616 19:57:51.329550 7954 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4d4bfa06e04d2d8de1b13263001b1b (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260616 19:57:51.372586 8030 tablet_service.cc:1511] Processing CreateTablet for tablet 9918060aeb7f44c892f2ade3f2ac6841 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260616 19:57:51.422811 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9918060aeb7f44c892f2ade3f2ac6841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.374377 8029 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4d4bfa06e04d2d8de1b13263001b1b (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260616 19:57:51.428087 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4d4bfa06e04d2d8de1b13263001b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.433384 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f64cb36408849c989ecda8d19ec4269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.441417 8159 tablet_bootstrap.cc:492] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:51.441895 7960 tablet_service.cc:1511] Processing CreateTablet for tablet 5eef7d39858d46729e29a621de3bc3dd (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260616 19:57:51.443332 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eef7d39858d46729e29a621de3bc3dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.453172 7957 tablet_service.cc:1511] Processing CreateTablet for tablet 56c25ff952f14593ad9cdf3cf561a445 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260616 19:57:51.454707 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c25ff952f14593ad9cdf3cf561a445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.456539 7952 tablet_service.cc:1511] Processing CreateTablet for tablet f24cd55ee50d4bcfaee34b2adaa9b08a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260616 19:57:51.457989 7952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24cd55ee50d4bcfaee34b2adaa9b08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.461942 8029 tablet_service.cc:1511] Processing CreateTablet for tablet f811e1e045a942fa8b77b5c1e2802efd (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260616 19:57:51.462527 7960 tablet_service.cc:1511] Processing CreateTablet for tablet 69c80791d0c44cb0b616bf402ae91b46 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260616 19:57:51.437661 8100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 303be6b16595410aae7147c6026435ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.473549 7953 tablet_service.cc:1511] Processing CreateTablet for tablet e22f98752ef9436aa8a128756832bff0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260616 19:57:51.474642 8101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.486095 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f811e1e045a942fa8b77b5c1e2802efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.497253 8031 tablet_service.cc:1511] Processing CreateTablet for tablet f24cd55ee50d4bcfaee34b2adaa9b08a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260616 19:57:51.498979 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24cd55ee50d4bcfaee34b2adaa9b08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.505960 8029 tablet_service.cc:1511] Processing CreateTablet for tablet e22f98752ef9436aa8a128756832bff0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260616 19:57:51.508699 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22f98752ef9436aa8a128756832bff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.515892 8159 tablet_bootstrap.cc:654] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:51.543291 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22f98752ef9436aa8a128756832bff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.521503 8031 tablet_service.cc:1511] Processing CreateTablet for tablet 5eef7d39858d46729e29a621de3bc3dd (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260616 19:57:51.548698 7954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4d4bfa06e04d2d8de1b13263001b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.549021 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eef7d39858d46729e29a621de3bc3dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.525972 8029 tablet_service.cc:1511] Processing CreateTablet for tablet 56c25ff952f14593ad9cdf3cf561a445 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260616 19:57:51.551965 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c25ff952f14593ad9cdf3cf561a445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.554051 7958 tablet_service.cc:1511] Processing CreateTablet for tablet c266418888004ab0a420a0dcbc5e8477 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260616 19:57:51.555590 7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c266418888004ab0a420a0dcbc5e8477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.557562 8035 tablet_service.cc:1511] Processing CreateTablet for tablet 69c80791d0c44cb0b616bf402ae91b46 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260616 19:57:51.559104 8035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c80791d0c44cb0b616bf402ae91b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.562006 7956 tablet_service.cc:1511] Processing CreateTablet for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260616 19:57:51.570544 8032 tablet_service.cc:1511] Processing CreateTablet for tablet c266418888004ab0a420a0dcbc5e8477 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260616 19:57:51.578318 7951 tablet_service.cc:1511] Processing CreateTablet for tablet 69dd551c580f43a4a5e348a7a31398e1 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260616 19:57:51.580890 8109 tablet_service.cc:1511] Processing CreateTablet for tablet e22f98752ef9436aa8a128756832bff0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260616 19:57:51.589146 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c266418888004ab0a420a0dcbc5e8477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.596553 8161 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:51.602507 8161 tablet_bootstrap.cc:654] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:51.590646 8105 tablet_service.cc:1511] Processing CreateTablet for tablet f24cd55ee50d4bcfaee34b2adaa9b08a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260616 19:57:51.591267 7956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.583873 8026 tablet_service.cc:1511] Processing CreateTablet for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260616 19:57:51.611100 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c80791d0c44cb0b616bf402ae91b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.612448 7951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69dd551c580f43a4a5e348a7a31398e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.606257 7955 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea1f7e6b1ac4325b6bc7a648cbba327 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260616 19:57:51.608209 8034 tablet_service.cc:1511] Processing CreateTablet for tablet 541911a4cee341d8911dc307cd4a7b44 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260616 19:57:51.616346 7955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1f7e6b1ac4325b6bc7a648cbba327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.619477 8033 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea1f7e6b1ac4325b6bc7a648cbba327 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260616 19:57:51.621352 8159 tablet_bootstrap.cc:492] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:51.621878 8159 ts_tablet_manager.cc:1403] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.181s user 0.019s sys 0.022s
I20260616 19:57:51.627590 8159 raft_consensus.cc:359] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.629902 8159 raft_consensus.cc:385] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:51.630772 8159 raft_consensus.cc:740] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:51.627552 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24cd55ee50d4bcfaee34b2adaa9b08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.633699 8030 tablet_service.cc:1511] Processing CreateTablet for tablet e8890d947e59455187876dccb01c8c06 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260616 19:57:51.632941 8159 consensus_queue.cc:260] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.636384 8109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22f98752ef9436aa8a128756832bff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.629181 8028 tablet_service.cc:1511] Processing CreateTablet for tablet 69dd551c580f43a4a5e348a7a31398e1 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260616 19:57:51.644853 8160 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:51.647388 7952 tablet_service.cc:1511] Processing CreateTablet for tablet f811e1e045a942fa8b77b5c1e2802efd (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260616 19:57:51.649514 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 5eef7d39858d46729e29a621de3bc3dd (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260616 19:57:51.656515 8104 tablet_service.cc:1511] Processing CreateTablet for tablet 56c25ff952f14593ad9cdf3cf561a445 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260616 19:57:51.658874 8107 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea1f7e6b1ac4325b6bc7a648cbba327 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260616 19:57:51.653770 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eef7d39858d46729e29a621de3bc3dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.664446 8034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 541911a4cee341d8911dc307cd4a7b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.663125 8108 tablet_service.cc:1511] Processing CreateTablet for tablet c266418888004ab0a420a0dcbc5e8477 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260616 19:57:51.665637 8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c266418888004ab0a420a0dcbc5e8477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.666772 7953 tablet_service.cc:1511] Processing CreateTablet for tablet e8890d947e59455187876dccb01c8c06 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260616 19:57:51.667591 8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1f7e6b1ac4325b6bc7a648cbba327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.668453 8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c25ff952f14593ad9cdf3cf561a445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.669871 8106 tablet_service.cc:1511] Processing CreateTablet for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260616 19:57:51.670431 8026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.649717 8027 tablet_service.cc:1511] Processing CreateTablet for tablet ba7d833edb374b10abaeb128d2084328 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260616 19:57:51.671305 8106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.671494 8027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba7d833edb374b10abaeb128d2084328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.673497 8107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1f7e6b1ac4325b6bc7a648cbba327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.675175 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8890d947e59455187876dccb01c8c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.677307 8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69dd551c580f43a4a5e348a7a31398e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.659210 7952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f811e1e045a942fa8b77b5c1e2802efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.689502 8159 ts_tablet_manager.cc:1434] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.067s user 0.009s sys 0.009s
I20260616 19:57:51.690351 7995 heartbeater.cc:499] Master 127.6.159.190:44129 was elected leader, sending a full tablet report...
W20260616 19:57:51.703061 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:51.704695 8159 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:51.675853 8100 tablet_service.cc:1511] Processing CreateTablet for tablet 69c80791d0c44cb0b616bf402ae91b46 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260616 19:57:51.709281 8100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c80791d0c44cb0b616bf402ae91b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.712306 8160 tablet_bootstrap.cc:654] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:51.716338 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8890d947e59455187876dccb01c8c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.728906 7960 tablet_service.cc:1511] Processing CreateTablet for tablet ba7d833edb374b10abaeb128d2084328 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260616 19:57:51.730405 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba7d833edb374b10abaeb128d2084328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.727567 8103 tablet_service.cc:1511] Processing CreateTablet for tablet f811e1e045a942fa8b77b5c1e2802efd (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260616 19:57:51.749778 8161 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:51.750295 8161 ts_tablet_manager.cc:1403] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.154s user 0.011s sys 0.005s
I20260616 19:57:51.764454 7959 tablet_service.cc:1511] Processing CreateTablet for tablet 541911a4cee341d8911dc307cd4a7b44 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260616 19:57:51.765789 7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 541911a4cee341d8911dc307cd4a7b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.703945 8101 tablet_service.cc:1511] Processing CreateTablet for tablet 69dd551c580f43a4a5e348a7a31398e1 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260616 19:57:51.775388 8109 tablet_service.cc:1511] Processing CreateTablet for tablet 541911a4cee341d8911dc307cd4a7b44 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260616 19:57:51.782059 8104 tablet_service.cc:1511] Processing CreateTablet for tablet e8890d947e59455187876dccb01c8c06 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260616 19:57:51.783540 8103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f811e1e045a942fa8b77b5c1e2802efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.786005 8102 tablet_service.cc:1511] Processing CreateTablet for tablet ba7d833edb374b10abaeb128d2084328 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260616 19:57:51.787312 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba7d833edb374b10abaeb128d2084328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.790304 8029 tablet_service.cc:1511] Processing CreateTablet for tablet 43378ff312f94246811895976a8715d6 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260616 19:57:51.791703 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43378ff312f94246811895976a8715d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.782069 7957 tablet_service.cc:1511] Processing CreateTablet for tablet 43378ff312f94246811895976a8715d6 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260616 19:57:51.787012 8161 raft_consensus.cc:359] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.801558 8161 raft_consensus.cc:385] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:51.801780 8101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69dd551c580f43a4a5e348a7a31398e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.801921 8161 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:51.803416 8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8890d947e59455187876dccb01c8c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.804929 8109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 541911a4cee341d8911dc307cd4a7b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.804098 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43378ff312f94246811895976a8715d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.805336 8159 tablet_bootstrap.cc:654] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:51.816442 8161 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.840472 8105 tablet_service.cc:1511] Processing CreateTablet for tablet 43378ff312f94246811895976a8715d6 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260616 19:57:51.877261 8160 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:51.877995 8160 ts_tablet_manager.cc:1403] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.234s user 0.007s sys 0.011s
I20260616 19:57:51.882540 7952 tablet_service.cc:1511] Processing CreateTablet for tablet 9da6a08976cb46d698d30af739dd2a3b (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260616 19:57:51.882714 8160 raft_consensus.cc:359] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.884007 7952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da6a08976cb46d698d30af739dd2a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.883381 8029 tablet_service.cc:1511] Processing CreateTablet for tablet 9da6a08976cb46d698d30af739dd2a3b (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260616 19:57:51.885390 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da6a08976cb46d698d30af739dd2a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.883411 8160 raft_consensus.cc:385] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:51.892976 8160 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:51.894137 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43378ff312f94246811895976a8715d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.894495 8160 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.909198 8144 heartbeater.cc:499] Master 127.6.159.190:44129 was elected leader, sending a full tablet report...
I20260616 19:57:51.910094 7953 tablet_service.cc:1511] Processing CreateTablet for tablet cdd9ba5e3e5c4292b330704225571321 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260616 19:57:51.910908 8160 ts_tablet_manager.cc:1434] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.033s user 0.006s sys 0.009s
I20260616 19:57:51.911459 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd9ba5e3e5c4292b330704225571321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.911667 8070 heartbeater.cc:499] Master 127.6.159.190:44129 was elected leader, sending a full tablet report...
I20260616 19:57:51.915748 8106 tablet_service.cc:1511] Processing CreateTablet for tablet 9da6a08976cb46d698d30af739dd2a3b (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260616 19:57:51.917342 8106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da6a08976cb46d698d30af739dd2a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.908989 8169 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:51.926496 8160 tablet_bootstrap.cc:492] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:51.926590 8032 tablet_service.cc:1511] Processing CreateTablet for tablet cdd9ba5e3e5c4292b330704225571321 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260616 19:57:51.927917 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd9ba5e3e5c4292b330704225571321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:51.932941 8160 tablet_bootstrap.cc:654] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:51.934270 8169 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.934852 8159 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:51.935381 8159 ts_tablet_manager.cc:1403] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.231s user 0.030s sys 0.012s
I20260616 19:57:51.938414 8159 raft_consensus.cc:359] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.939157 8159 raft_consensus.cc:385] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:51.939442 8159 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:51.940865 8159 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.942091 8169 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:51.957501 8166 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:51.958012 8166 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:51.960114 8166 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:51.970337 8160 tablet_bootstrap.cc:492] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:51.970912 8160 ts_tablet_manager.cc:1403] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.014s
I20260616 19:57:51.989615 7959 tablet_service.cc:1511] Processing CreateTablet for tablet 91d1874edf8b4ac187032e20c2e8d509 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260616 19:57:51.972565 8107 tablet_service.cc:1511] Processing CreateTablet for tablet cdd9ba5e3e5c4292b330704225571321 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260616 19:57:52.007184 7957 tablet_service.cc:1511] Processing CreateTablet for tablet 3a82eaaad5b1444b9ca242cd0aaeed47 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260616 19:57:52.009461 8103 tablet_service.cc:1511] Processing CreateTablet for tablet 3a82eaaad5b1444b9ca242cd0aaeed47 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260616 19:57:52.010754 8103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a82eaaad5b1444b9ca242cd0aaeed47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.016211 7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d1874edf8b4ac187032e20c2e8d509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.017660 7956 tablet_service.cc:1511] Processing CreateTablet for tablet eb7747b7f8144d689a26ea1780989278 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260616 19:57:52.021018 8105 tablet_service.cc:1511] Processing CreateTablet for tablet 91d1874edf8b4ac187032e20c2e8d509 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260616 19:57:52.022921 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d1874edf8b4ac187032e20c2e8d509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.028365 8161 ts_tablet_manager.cc:1434] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.278s user 0.061s sys 0.030s
I20260616 19:57:52.034924 8161 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:52.036674 7956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7747b7f8144d689a26ea1780989278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.037691 8107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd9ba5e3e5c4292b330704225571321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.040436 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a82eaaad5b1444b9ca242cd0aaeed47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.044842 8160 raft_consensus.cc:359] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.045598 8160 raft_consensus.cc:385] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.045888 8160 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.046624 8160 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.049722 8159 ts_tablet_manager.cc:1434] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.114s user 0.009s sys 0.001s
I20260616 19:57:52.050720 8159 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:52.061297 8101 tablet_service.cc:1511] Processing CreateTablet for tablet eb7747b7f8144d689a26ea1780989278 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260616 19:57:52.062727 8101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7747b7f8144d689a26ea1780989278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.072068 8160 ts_tablet_manager.cc:1434] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.101s user 0.009s sys 0.000s
I20260616 19:57:52.074384 8169 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.074275 8033 tablet_service.cc:1511] Processing CreateTablet for tablet 3a82eaaad5b1444b9ca242cd0aaeed47 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260616 19:57:52.074612 8026 tablet_service.cc:1511] Processing CreateTablet for tablet eb7747b7f8144d689a26ea1780989278 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260616 19:57:52.074889 8169 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.075629 8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a82eaaad5b1444b9ca242cd0aaeed47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.076237 8026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7747b7f8144d689a26ea1780989278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.077086 8169 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:52.078863 8029 tablet_service.cc:1511] Processing CreateTablet for tablet 91d1874edf8b4ac187032e20c2e8d509 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260616 19:57:52.080118 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d1874edf8b4ac187032e20c2e8d509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.083014 7951 tablet_service.cc:1511] Processing CreateTablet for tablet 33c17f5e21da4e9893d22581a43ceafb (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260616 19:57:52.084335 7951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33c17f5e21da4e9893d22581a43ceafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.089905 8164 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.090401 8164 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.073155 8160 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:52.094835 8164 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:52.095178 8161 tablet_bootstrap.cc:654] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.111361 8108 tablet_service.cc:1511] Processing CreateTablet for tablet bd096493918740ac8b196f47037f5dc5 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260616 19:57:52.145396 8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd096493918740ac8b196f47037f5dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.151319 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 33c17f5e21da4e9893d22581a43ceafb (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260616 19:57:52.152932 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33c17f5e21da4e9893d22581a43ceafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.153359 8030 tablet_service.cc:1511] Processing CreateTablet for tablet 33c17f5e21da4e9893d22581a43ceafb (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260616 19:57:52.154597 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33c17f5e21da4e9893d22581a43ceafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.160081 8164 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.160215 8035 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffc56f0158c4f8a99ad2d5a819bde34 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260616 19:57:52.160611 8164 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.161578 8035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffc56f0158c4f8a99ad2d5a819bde34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.162528 8164 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:52.160215 8034 tablet_service.cc:1511] Processing CreateTablet for tablet bd096493918740ac8b196f47037f5dc5 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260616 19:57:52.170909 8034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd096493918740ac8b196f47037f5dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.168577 8104 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffc56f0158c4f8a99ad2d5a819bde34 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260616 19:57:52.174917 8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffc56f0158c4f8a99ad2d5a819bde34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.168414 8159 tablet_bootstrap.cc:654] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.169049 8100 tablet_service.cc:1511] Processing CreateTablet for tablet 02d9effe013f408fad87267b74d4e1ac (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260616 19:57:52.177583 8100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d9effe013f408fad87267b74d4e1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.197908 8031 tablet_service.cc:1511] Processing CreateTablet for tablet e9e060ef9a144de99b92bcba80c31176 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260616 19:57:52.199435 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e060ef9a144de99b92bcba80c31176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.210331 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:52.211215 8045 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:52.213603 8028 tablet_service.cc:1511] Processing CreateTablet for tablet f52f3e8cd74b471e9c8b0fc90839f7b0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260616 19:57:52.215685 8084 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:52.216857 8166 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:52.217242 8166 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:52.217577 8166 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:52.220502 8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52f3e8cd74b471e9c8b0fc90839f7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.225318 8160 tablet_bootstrap.cc:654] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.233083 7953 tablet_service.cc:1511] Processing CreateTablet for tablet bd096493918740ac8b196f47037f5dc5 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260616 19:57:52.233356 7957 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffc56f0158c4f8a99ad2d5a819bde34 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260616 19:57:52.234632 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd096493918740ac8b196f47037f5dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.235380 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:52.236179 7960 tablet_service.cc:1511] Processing CreateTablet for tablet f52f3e8cd74b471e9c8b0fc90839f7b0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260616 19:57:52.235850 7952 tablet_service.cc:1511] Processing CreateTablet for tablet e9e060ef9a144de99b92bcba80c31176 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260616 19:57:52.236804 7956 tablet_service.cc:1511] Processing CreateTablet for tablet 02d9effe013f408fad87267b74d4e1ac (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260616 19:57:52.237386 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:52.239471 8011 leader_election.cc:343] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.240571 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52f3e8cd74b471e9c8b0fc90839f7b0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260616 19:57:52.240911 8011 leader_election.cc:343] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.244891 7952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e060ef9a144de99b92bcba80c31176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.248292 7956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d9effe013f408fad87267b74d4e1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.249537 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:52.250613 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffc56f0158c4f8a99ad2d5a819bde34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.250751 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:57:52.252498 8009 leader_election.cc:343] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.252967 8009 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:52.254081 8169 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:52.256839 8109 tablet_service.cc:1511] Processing CreateTablet for tablet e9e060ef9a144de99b92bcba80c31176 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260616 19:57:52.268468 8032 tablet_service.cc:1511] Processing CreateTablet for tablet 02d9effe013f408fad87267b74d4e1ac (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260616 19:57:52.269845 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d9effe013f408fad87267b74d4e1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.270889 8159 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:52.271404 8159 ts_tablet_manager.cc:1403] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.221s user 0.006s sys 0.022s
I20260616 19:57:52.274405 8159 raft_consensus.cc:359] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.275139 8159 raft_consensus.cc:385] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.275434 8159 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.263207 8109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e060ef9a144de99b92bcba80c31176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.281533 7958 tablet_service.cc:1511] Processing CreateTablet for tablet 9837c086539a4592bb5aada5872652ca (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260616 19:57:52.282825 7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9837c086539a4592bb5aada5872652ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.294327 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:52.294894 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:52.296257 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:52.297917 7935 leader_election.cc:343] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
W20260616 19:57:52.298595 7935 leader_election.cc:343] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.301344 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:52.301541 8161 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:52.302350 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:52.256588 8105 tablet_service.cc:1511] Processing CreateTablet for tablet f52f3e8cd74b471e9c8b0fc90839f7b0 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260616 19:57:52.303894 7934 leader_election.cc:343] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.304360 8161 ts_tablet_manager.cc:1403] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.270s user 0.016s sys 0.005s
I20260616 19:57:52.304399 7934 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:52.305320 8164 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260616 19:57:52.306560 7934 leader_election.cc:343] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.307003 7934 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:52.307689 8164 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:52.307418 8161 raft_consensus.cc:359] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.316962 8161 raft_consensus.cc:385] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.317293 8161 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.317799 8159 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.318056 8161 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.304404 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52f3e8cd74b471e9c8b0fc90839f7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.278936 8166 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.324802 8119 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:52.326684 8030 tablet_service.cc:1511] Processing CreateTablet for tablet 9837c086539a4592bb5aada5872652ca (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260616 19:57:52.328063 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9837c086539a4592bb5aada5872652ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.327991 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 9837c086539a4592bb5aada5872652ca (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260616 19:57:52.328467 8009 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:52.329622 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9837c086539a4592bb5aada5872652ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.329695 8169 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:52.337968 8166 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:52.338284 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:52.339027 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:52.339769 8169 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:52.346174 8164 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.348419 8164 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.349561 7958 tablet_service.cc:1511] Processing CreateTablet for tablet 00bf27a79bc04305bb057b972a08c316 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260616 19:57:52.352329 7969 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:52.354482 8044 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:52.353689 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:52.355939 8045 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:52.356288 8164 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:52.357139 8084 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:52.357400 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:52.358173 8119 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:52.359226 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:52.360277 7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00bf27a79bc04305bb057b972a08c316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.364085 8108 tablet_service.cc:1511] Processing CreateTablet for tablet 00bf27a79bc04305bb057b972a08c316 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260616 19:57:52.365536 8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00bf27a79bc04305bb057b972a08c316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.361022 8164 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:52.367597 8166 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:52.368201 8166 raft_consensus.cc:697] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:52.369103 8166 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.372609 8028 tablet_service.cc:1511] Processing CreateTablet for tablet 0d19ee585ead4942a40096f7aef3ba50 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260616 19:57:52.375150 8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d19ee585ead4942a40096f7aef3ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.364097 8160 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:52.376372 8160 ts_tablet_manager.cc:1403] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.303s user 0.013s sys 0.002s
I20260616 19:57:52.373067 8034 tablet_service.cc:1511] Processing CreateTablet for tablet 00bf27a79bc04305bb057b972a08c316 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260616 19:57:52.388054 7957 tablet_service.cc:1511] Processing CreateTablet for tablet 0d19ee585ead4942a40096f7aef3ba50 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260616 19:57:52.390118 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d19ee585ead4942a40096f7aef3ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.395345 8105 tablet_service.cc:1511] Processing CreateTablet for tablet 0d19ee585ead4942a40096f7aef3ba50 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260616 19:57:52.398329 8034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00bf27a79bc04305bb057b972a08c316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.399326 8161 ts_tablet_manager.cc:1434] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.095s user 0.010s sys 0.002s
I20260616 19:57:52.400506 8161 tablet_bootstrap.cc:492] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:52.396764 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d19ee585ead4942a40096f7aef3ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.403304 7866 catalog_manager.cc:5696] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:52.410573 8159 ts_tablet_manager.cc:1434] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.139s user 0.008s sys 0.004s
I20260616 19:57:52.411644 8159 tablet_bootstrap.cc:492] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:52.412909 8031 tablet_service.cc:1511] Processing CreateTablet for tablet 138ad2f3a7f24eb8969d70eceea9ed3f (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260616 19:57:52.414201 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 138ad2f3a7f24eb8969d70eceea9ed3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.415767 7959 tablet_service.cc:1511] Processing CreateTablet for tablet 138ad2f3a7f24eb8969d70eceea9ed3f (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260616 19:57:52.417107 7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 138ad2f3a7f24eb8969d70eceea9ed3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.416802 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 138ad2f3a7f24eb8969d70eceea9ed3f (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260616 19:57:52.418041 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 138ad2f3a7f24eb8969d70eceea9ed3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.396090 8164 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:52.423199 7956 tablet_service.cc:1511] Processing CreateTablet for tablet b12fbb557a1a4c488d787ecb4e750fad (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260616 19:57:52.429091 7956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12fbb557a1a4c488d787ecb4e750fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.431468 8030 tablet_service.cc:1511] Processing CreateTablet for tablet b12fbb557a1a4c488d787ecb4e750fad (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260616 19:57:52.435981 7953 tablet_service.cc:1511] Processing CreateTablet for tablet 5ab514ead9d34ff598a454bd28675d12 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260616 19:57:52.437250 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ab514ead9d34ff598a454bd28675d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.445461 8032 tablet_service.cc:1511] Processing CreateTablet for tablet 5ab514ead9d34ff598a454bd28675d12 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260616 19:57:52.448283 7960 tablet_service.cc:1511] Processing CreateTablet for tablet 6c06fb426fd349ac813aef7f33c47e85 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260616 19:57:52.449566 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c06fb426fd349ac813aef7f33c47e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.453244 8100 tablet_service.cc:1511] Processing CreateTablet for tablet 5ab514ead9d34ff598a454bd28675d12 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260616 19:57:52.454568 8100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ab514ead9d34ff598a454bd28675d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.457051 8160 raft_consensus.cc:359] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.457840 8160 raft_consensus.cc:385] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.458125 8160 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.441780 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12fbb557a1a4c488d787ecb4e750fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.460399 8161 tablet_bootstrap.cc:654] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.460764 8159 tablet_bootstrap.cc:654] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.460764 8109 tablet_service.cc:1511] Processing CreateTablet for tablet b12fbb557a1a4c488d787ecb4e750fad (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260616 19:57:52.464813 8109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12fbb557a1a4c488d787ecb4e750fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.467620 8035 tablet_service.cc:1511] Processing CreateTablet for tablet 6c06fb426fd349ac813aef7f33c47e85 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260616 19:57:52.468840 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ab514ead9d34ff598a454bd28675d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.469699 8035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c06fb426fd349ac813aef7f33c47e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.480985 8160 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.484714 7958 tablet_service.cc:1511] Processing CreateTablet for tablet cf8a24d3c225486fa9270926ffe6a40e (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260616 19:57:52.494400 8028 tablet_service.cc:1511] Processing CreateTablet for tablet cf8a24d3c225486fa9270926ffe6a40e (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260616 19:57:52.506070 8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf8a24d3c225486fa9270926ffe6a40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.507295 8160 ts_tablet_manager.cc:1434] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.131s user 0.005s sys 0.002s
I20260616 19:57:52.508680 7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf8a24d3c225486fa9270926ffe6a40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.515113 8108 tablet_service.cc:1511] Processing CreateTablet for tablet 6c06fb426fd349ac813aef7f33c47e85 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260616 19:57:52.516919 8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c06fb426fd349ac813aef7f33c47e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.530725 8160 tablet_bootstrap.cc:492] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:52.540773 7953 tablet_service.cc:1511] Processing CreateTablet for tablet bd16983859704e7fb1ce248d36c2a8bf (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260616 19:57:52.541086 8031 tablet_service.cc:1511] Processing CreateTablet for tablet bd16983859704e7fb1ce248d36c2a8bf (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260616 19:57:52.542150 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd16983859704e7fb1ce248d36c2a8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.542364 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd16983859704e7fb1ce248d36c2a8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.548588 8102 tablet_service.cc:1511] Processing CreateTablet for tablet bd16983859704e7fb1ce248d36c2a8bf (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260616 19:57:52.549942 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd16983859704e7fb1ce248d36c2a8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.550523 8161 tablet_bootstrap.cc:492] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:52.551090 8161 ts_tablet_manager.cc:1403] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.151s user 0.024s sys 0.029s
I20260616 19:57:52.553719 8159 tablet_bootstrap.cc:492] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:52.554201 8159 ts_tablet_manager.cc:1403] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.143s user 0.009s sys 0.009s
I20260616 19:57:52.556027 8160 tablet_bootstrap.cc:654] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.558566 8105 tablet_service.cc:1511] Processing CreateTablet for tablet cf8a24d3c225486fa9270926ffe6a40e (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260616 19:57:52.570722 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf8a24d3c225486fa9270926ffe6a40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.554229 8161 raft_consensus.cc:359] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.571259 8161 raft_consensus.cc:385] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.571612 8161 raft_consensus.cc:740] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.572325 8035 tablet_service.cc:1511] Processing CreateTablet for tablet b14a29bdfa9541db9d3f1e93a225a342 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260616 19:57:52.572641 8161 consensus_queue.cc:260] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.573626 8035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b14a29bdfa9541db9d3f1e93a225a342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.575405 8161 ts_tablet_manager.cc:1434] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.024s user 0.003s sys 0.003s
I20260616 19:57:52.576522 8161 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:52.576289 7956 tablet_service.cc:1511] Processing CreateTablet for tablet b14a29bdfa9541db9d3f1e93a225a342 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260616 19:57:52.559623 8159 raft_consensus.cc:359] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.577142 8159 raft_consensus.cc:385] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.577594 7956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b14a29bdfa9541db9d3f1e93a225a342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.577497 8159 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.578527 8159 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.581142 8159 ts_tablet_manager.cc:1434] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.027s user 0.001s sys 0.005s
I20260616 19:57:52.584972 8159 tablet_bootstrap.cc:492] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:52.589399 8109 tablet_service.cc:1511] Processing CreateTablet for tablet b14a29bdfa9541db9d3f1e93a225a342 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260616 19:57:52.590754 8109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b14a29bdfa9541db9d3f1e93a225a342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.597960 7958 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0ed6745c784554bc3af153bcda098a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260616 19:57:52.598600 8161 tablet_bootstrap.cc:654] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.599287 7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0ed6745c784554bc3af153bcda098a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.599313 8032 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0ed6745c784554bc3af153bcda098a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260616 19:57:52.600991 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0ed6745c784554bc3af153bcda098a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.602142 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0ed6745c784554bc3af153bcda098a (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260616 19:57:52.607976 7960 tablet_service.cc:1511] Processing CreateTablet for tablet b96ebb721a6a429ca8c0d058adaf5c6f (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260616 19:57:52.609294 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96ebb721a6a429ca8c0d058adaf5c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.616612 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0ed6745c784554bc3af153bcda098a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.626688 7957 tablet_service.cc:1511] Processing CreateTablet for tablet d06bfb4102944cae932c8710569e99d3 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260616 19:57:52.626668 8100 tablet_service.cc:1511] Processing CreateTablet for tablet b96ebb721a6a429ca8c0d058adaf5c6f (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260616 19:57:52.622473 8030 tablet_service.cc:1511] Processing CreateTablet for tablet b96ebb721a6a429ca8c0d058adaf5c6f (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260616 19:57:52.628355 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06bfb4102944cae932c8710569e99d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.629122 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96ebb721a6a429ca8c0d058adaf5c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.632556 8100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b96ebb721a6a429ca8c0d058adaf5c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.628262 8186 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.636445 8186 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.638720 8028 tablet_service.cc:1511] Processing CreateTablet for tablet d06bfb4102944cae932c8710569e99d3 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260616 19:57:52.639513 8186 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:57:52.640072 8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06bfb4102944cae932c8710569e99d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.639828 8107 tablet_service.cc:1511] Processing CreateTablet for tablet d06bfb4102944cae932c8710569e99d3 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260616 19:57:52.641074 8107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06bfb4102944cae932c8710569e99d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.644778 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:52.645452 7969 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:52.646446 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:52.647845 8084 leader_election.cc:343] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.650697 8085 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:52.651578 8186 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:52.651921 8186 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:52.652300 8186 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:52.656760 7959 tablet_service.cc:1511] Processing CreateTablet for tablet 2e660e68b6c54e9095967ae4b226b0d2 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260616 19:57:52.658540 8034 tablet_service.cc:1511] Processing CreateTablet for tablet 2e660e68b6c54e9095967ae4b226b0d2 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260616 19:57:52.658974 7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e660e68b6c54e9095967ae4b226b0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.659862 8034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e660e68b6c54e9095967ae4b226b0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.674567 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 2e660e68b6c54e9095967ae4b226b0d2 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260616 19:57:52.675763 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e660e68b6c54e9095967ae4b226b0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.676905 8186 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.679724 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:52.682229 8160 tablet_bootstrap.cc:492] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:52.682718 8160 ts_tablet_manager.cc:1403] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.152s user 0.018s sys 0.027s
I20260616 19:57:52.685621 8160 raft_consensus.cc:359] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.686371 8160 raft_consensus.cc:385] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.686741 8160 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
W20260616 19:57:52.687490 8084 leader_election.cc:343] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.688438 8160 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.690351 8186 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:57:52.690390 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:52.691118 7969 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:52.693574 7953 tablet_service.cc:1511] Processing CreateTablet for tablet 38333e3e28e64678b975f1c136bcd790 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260616 19:57:52.694969 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38333e3e28e64678b975f1c136bcd790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.707939 8159 tablet_bootstrap.cc:654] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:52.708858 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:52.709022 8032 tablet_service.cc:1511] Processing CreateTablet for tablet 38333e3e28e64678b975f1c136bcd790 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260616 19:57:52.712869 8106 tablet_service.cc:1511] Processing CreateTablet for tablet 38333e3e28e64678b975f1c136bcd790 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260616 19:57:52.713795 7957 tablet_service.cc:1511] Processing CreateTablet for tablet 9b74ddd432434f15bc1a4a3141d4c6aa (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260616 19:57:52.719480 8106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38333e3e28e64678b975f1c136bcd790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.720928 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38333e3e28e64678b975f1c136bcd790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.725997 8035 tablet_service.cc:1511] Processing CreateTablet for tablet 9b74ddd432434f15bc1a4a3141d4c6aa (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260616 19:57:52.727619 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 9b74ddd432434f15bc1a4a3141d4c6aa (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260616 19:57:52.734274 8161 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:52.734395 7959 tablet_service.cc:1511] Processing CreateTablet for tablet ecd3baa2a052460bb18e02978cb8b80d (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260616 19:57:52.735862 8035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b74ddd432434f15bc1a4a3141d4c6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.735528 8031 tablet_service.cc:1511] Processing CreateTablet for tablet ecd3baa2a052460bb18e02978cb8b80d (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260616 19:57:52.737172 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecd3baa2a052460bb18e02978cb8b80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.734817 8161 ts_tablet_manager.cc:1403] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.159s user 0.005s sys 0.010s
I20260616 19:57:52.739012 8101 tablet_service.cc:1511] Processing CreateTablet for tablet ecd3baa2a052460bb18e02978cb8b80d (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260616 19:57:52.742692 7960 tablet_service.cc:1511] Processing CreateTablet for tablet 3dc4c19625e84ce29cb4ca6993524bf1 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260616 19:57:52.744601 8186 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.744786 8029 tablet_service.cc:1511] Processing CreateTablet for tablet 3dc4c19625e84ce29cb4ca6993524bf1 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260616 19:57:52.745088 8186 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.746073 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc4c19625e84ce29cb4ca6993524bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.747220 8186 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:52.750573 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b74ddd432434f15bc1a4a3141d4c6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.745111 8160 ts_tablet_manager.cc:1434] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.062s user 0.007s sys 0.001s
I20260616 19:57:52.751180 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc4c19625e84ce29cb4ca6993524bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.752238 7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecd3baa2a052460bb18e02978cb8b80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.752166 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:52.752400 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:52.753072 8045 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:52.754297 8084 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:52.755122 8186 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:52.755466 8186 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:52.755779 8186 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:52.755923 8085 leader_election.cc:343] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.760828 8101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecd3baa2a052460bb18e02978cb8b80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.762084 8160 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:52.764907 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b74ddd432434f15bc1a4a3141d4c6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.741465 8161 raft_consensus.cc:359] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.778503 8107 tablet_service.cc:1511] Processing CreateTablet for tablet 3dc4c19625e84ce29cb4ca6993524bf1 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260616 19:57:52.779851 8107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc4c19625e84ce29cb4ca6993524bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.781555 7953 tablet_service.cc:1511] Processing CreateTablet for tablet 3e92f5df5ec84f069decf29684f08b06 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260616 19:57:52.782668 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e92f5df5ec84f069decf29684f08b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.775628 8161 raft_consensus.cc:385] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.800143 8161 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.815482 7959 tablet_service.cc:1511] Processing CreateTablet for tablet 30950534b1834f6886db15f85f0abf14 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260616 19:57:52.816480 8169 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.816854 7959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30950534b1834f6886db15f85f0abf14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.816942 8169 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.819049 8169 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:52.819289 8034 tablet_service.cc:1511] Processing CreateTablet for tablet 2bf8afc79e3c417c814ac2fb57557096 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260616 19:57:52.820571 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:52.821164 7957 tablet_service.cc:1511] Processing CreateTablet for tablet 43900f33cd8249e2a494e6f9dff7b254 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260616 19:57:52.822252 7957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43900f33cd8249e2a494e6f9dff7b254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.823971 8029 tablet_service.cc:1511] Processing CreateTablet for tablet 30950534b1834f6886db15f85f0abf14 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260616 19:57:52.823952 7960 tablet_service.cc:1511] Processing CreateTablet for tablet 2bf8afc79e3c417c814ac2fb57557096 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260616 19:57:52.825167 7960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf8afc79e3c417c814ac2fb57557096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.825861 8160 tablet_bootstrap.cc:654] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:52.822254 8011 leader_election.cc:343] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.827903 8161 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.828640 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:57:52.830022 8009 leader_election.cc:343] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.830482 8009 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:52.830941 8161 ts_tablet_manager.cc:1434] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.092s user 0.007s sys 0.000s
I20260616 19:57:52.820565 8034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf8afc79e3c417c814ac2fb57557096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.831058 8032 tablet_service.cc:1511] Processing CreateTablet for tablet 3e92f5df5ec84f069decf29684f08b06 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260616 19:57:52.832346 8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e92f5df5ec84f069decf29684f08b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.832589 8169 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:52.811967 7969 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:52.836470 8108 tablet_service.cc:1511] Processing CreateTablet for tablet 3e92f5df5ec84f069decf29684f08b06 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260616 19:57:52.837530 8085 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:52.837798 8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e92f5df5ec84f069decf29684f08b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.842695 8105 tablet_service.cc:1511] Processing CreateTablet for tablet 30950534b1834f6886db15f85f0abf14 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260616 19:57:52.843976 8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30950534b1834f6886db15f85f0abf14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.844355 8166 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:52.845934 8031 tablet_service.cc:1511] Processing CreateTablet for tablet 43900f33cd8249e2a494e6f9dff7b254 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260616 19:57:52.847179 8102 tablet_service.cc:1511] Processing CreateTablet for tablet 2bf8afc79e3c417c814ac2fb57557096 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260616 19:57:52.848896 8104 tablet_service.cc:1511] Processing CreateTablet for tablet 43900f33cd8249e2a494e6f9dff7b254 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260616 19:57:52.852118 8028 tablet_service.cc:1511] Processing CreateTablet for tablet 87bfd6ef4d304dc9899bf6fbd1260a11 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260616 19:57:52.865078 8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43900f33cd8249e2a494e6f9dff7b254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.868103 8109 tablet_service.cc:1511] Processing CreateTablet for tablet 87bfd6ef4d304dc9899bf6fbd1260a11 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260616 19:57:52.869311 8159 tablet_bootstrap.cc:492] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:52.869689 8109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87bfd6ef4d304dc9899bf6fbd1260a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.869844 8159 ts_tablet_manager.cc:1403] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.285s user 0.013s sys 0.031s
I20260616 19:57:52.873238 8102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf8afc79e3c417c814ac2fb57557096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.874943 8186 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.876807 7958 tablet_service.cc:1511] Processing CreateTablet for tablet de0955b8f32b489d9b813e4d4ea2cf39 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260616 19:57:52.878073 7958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0955b8f32b489d9b813e4d4ea2cf39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.878188 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:57:52.880141 8085 leader_election.cc:343] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.858302 8161 tablet_bootstrap.cc:492] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:52.859143 8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30950534b1834f6886db15f85f0abf14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.860334 8164 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:52.883594 8164 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.885155 8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87bfd6ef4d304dc9899bf6fbd1260a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.885862 8164 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:52.887166 8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43900f33cd8249e2a494e6f9dff7b254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.890134 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:52.887596 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:52.892259 7935 leader_election.cc:343] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.873250 8159 raft_consensus.cc:359] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.893002 8159 raft_consensus.cc:385] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.893330 8159 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
W20260616 19:57:52.893106 7934 leader_election.cc:343] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:52.894003 7934 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:52.894084 8159 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.895412 8164 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:52.901654 8030 tablet_service.cc:1511] Processing CreateTablet for tablet de0955b8f32b489d9b813e4d4ea2cf39 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260616 19:57:52.903306 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0955b8f32b489d9b813e4d4ea2cf39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.918238 8160 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:52.918751 8160 ts_tablet_manager.cc:1403] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.157s user 0.013s sys 0.003s
I20260616 19:57:52.917102 8199 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260616 19:57:52.921491 8101 tablet_service.cc:1511] Processing CreateTablet for tablet de0955b8f32b489d9b813e4d4ea2cf39 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260616 19:57:52.921694 8160 raft_consensus.cc:359] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.936888 8160 raft_consensus.cc:385] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:52.937280 8160 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:52.938577 8161 tablet_bootstrap.cc:654] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:52.938143 8160 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.917141 7953 tablet_service.cc:1511] Processing CreateTablet for tablet 87bfd6ef4d304dc9899bf6fbd1260a11 (DEFAULT_TABLE table=test-workload [id=274d51639c084ed39f0f2badc5040b90]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260616 19:57:52.940852 7953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87bfd6ef4d304dc9899bf6fbd1260a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.925345 8186 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:52.925935 8166 raft_consensus.cc:697] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:52.926728 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:52.945298 8044 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:52.948556 8101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0955b8f32b489d9b813e4d4ea2cf39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:57:52.951967 8166 consensus_queue.cc:237] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:52.960131 8071 tablet.cc:2406] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260616 19:57:52.963001 8160 ts_tablet_manager.cc:1434] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.044s user 0.008s sys 0.001s
I20260616 19:57:52.964035 8160 tablet_bootstrap.cc:492] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:52.966216 8044 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:52.967492 8084 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:52.971653 8186 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:52.973932 8186 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:52.975196 8159 ts_tablet_manager.cc:1434] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.105s user 0.003s sys 0.011s
I20260616 19:57:52.974736 8186 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:52.981561 8159 tablet_bootstrap.cc:492] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:52.988435 8159 tablet_bootstrap.cc:654] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.006985 8199 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260616 19:57:53.007300 8160 tablet_bootstrap.cc:654] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.037748 8169 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.038273 8169 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.042191 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:53.043555 8011 leader_election.cc:343] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.044144 8169 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.045289 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:57:53.046531 8009 leader_election.cc:343] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.046989 8009 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:53.048583 8169 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.049180 8161 tablet_bootstrap.cc:492] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.049635 8161 ts_tablet_manager.cc:1403] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.192s user 0.021s sys 0.028s
I20260616 19:57:53.050570 7867 catalog_manager.cc:5696] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:53.051625 8161 raft_consensus.cc:359] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.052906 8161 raft_consensus.cc:385] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.053251 8161 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.053555 8159 tablet_bootstrap.cc:492] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.054031 8159 ts_tablet_manager.cc:1403] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.073s user 0.011s sys 0.007s
I20260616 19:57:53.054105 8161 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.056492 8161 ts_tablet_manager.cc:1434] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260616 19:57:53.057422 8159 raft_consensus.cc:359] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.057874 8161 tablet_bootstrap.cc:492] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.058161 8159 raft_consensus.cc:385] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.058458 8159 raft_consensus.cc:740] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.061905 8159 consensus_queue.cc:260] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.064178 8159 ts_tablet_manager.cc:1434] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260616 19:57:53.064989 8161 tablet_bootstrap.cc:654] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.065600 8159 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.074353 8159 tablet_bootstrap.cc:654] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.074875 8160 tablet_bootstrap.cc:492] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.075373 8160 ts_tablet_manager.cc:1403] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.112s user 0.005s sys 0.020s
I20260616 19:57:53.076570 8199 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.077061 8199 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.078378 8160 raft_consensus.cc:359] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.079087 8160 raft_consensus.cc:385] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.079360 8160 raft_consensus.cc:740] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.080068 8160 consensus_queue.cc:260] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.080840 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.081431 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:53.082571 8085 leader_election.cc:343] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.082746 8160 ts_tablet_manager.cc:1434] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
W20260616 19:57:53.083542 8084 leader_election.cc:343] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.084192 8084 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.086015 8208 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.087059 8199 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.089964 8160 tablet_bootstrap.cc:492] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.090868 8159 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.091391 8159 ts_tablet_manager.cc:1403] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.005s
I20260616 19:57:53.092937 8161 tablet_bootstrap.cc:492] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.093410 8161 ts_tablet_manager.cc:1403] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.036s user 0.013s sys 0.004s
I20260616 19:57:53.095060 8159 raft_consensus.cc:359] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.095804 8159 raft_consensus.cc:385] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.096115 8159 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.096670 8160 tablet_bootstrap.cc:654] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.096382 8161 raft_consensus.cc:359] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.097098 8161 raft_consensus.cc:385] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.097024 8159 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.097536 8161 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.098409 8161 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.099429 8159 ts_tablet_manager.cc:1434] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260616 19:57:53.100632 8159 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.101020 8161 ts_tablet_manager.cc:1434] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260616 19:57:53.102411 8161 tablet_bootstrap.cc:492] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.107765 8159 tablet_bootstrap.cc:654] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.109190 8161 tablet_bootstrap.cc:654] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.109226 8160 tablet_bootstrap.cc:492] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.110307 8160 ts_tablet_manager.cc:1403] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.001s
I20260616 19:57:53.109073 7867 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:53.113767 8160 raft_consensus.cc:359] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.114497 8160 raft_consensus.cc:385] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.114784 8160 raft_consensus.cc:740] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.115582 8160 consensus_queue.cc:260] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.120937 8160 ts_tablet_manager.cc:1434] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260616 19:57:53.121878 8160 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.128314 8160 tablet_bootstrap.cc:654] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.148720 8160 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.149209 8160 ts_tablet_manager.cc:1403] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.004s
I20260616 19:57:53.152230 8160 raft_consensus.cc:359] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.152894 8160 raft_consensus.cc:385] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.153169 8160 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.153877 8159 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.154371 8159 ts_tablet_manager.cc:1403] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.054s user 0.034s sys 0.019s
I20260616 19:57:53.153864 8160 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.154795 8161 tablet_bootstrap.cc:492] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.155287 8161 ts_tablet_manager.cc:1403] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.053s user 0.022s sys 0.028s
I20260616 19:57:53.156009 8160 ts_tablet_manager.cc:1434] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260616 19:57:53.157148 8160 tablet_bootstrap.cc:492] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.156911 8159 raft_consensus.cc:359] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.157590 8159 raft_consensus.cc:385] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.157886 8159 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.158336 8161 raft_consensus.cc:359] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.159093 8161 raft_consensus.cc:385] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.158643 8159 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.159507 8161 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.160507 8161 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.162743 8161 ts_tablet_manager.cc:1434] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260616 19:57:53.163769 8161 tablet_bootstrap.cc:492] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.163995 8160 tablet_bootstrap.cc:654] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.170938 8161 tablet_bootstrap.cc:654] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.172103 8160 tablet_bootstrap.cc:492] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.172600 8160 ts_tablet_manager.cc:1403] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260616 19:57:53.176313 8159 ts_tablet_manager.cc:1434] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20260616 19:57:53.177307 8159 tablet_bootstrap.cc:492] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.180269 8160 raft_consensus.cc:359] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.180791 8161 tablet_bootstrap.cc:492] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.181061 8160 raft_consensus.cc:385] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.181435 8160 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.181605 8161 ts_tablet_manager.cc:1403] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.003s
I20260616 19:57:53.182126 8160 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.182695 8159 tablet_bootstrap.cc:654] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.184203 8160 ts_tablet_manager.cc:1434] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260616 19:57:53.184505 8161 raft_consensus.cc:359] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.185137 8161 raft_consensus.cc:385] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.185276 8160 tablet_bootstrap.cc:492] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.185400 8161 raft_consensus.cc:740] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.186142 8161 consensus_queue.cc:260] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.191840 8161 ts_tablet_manager.cc:1434] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260616 19:57:53.192682 8160 tablet_bootstrap.cc:654] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.192978 8161 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.193151 8169 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.193640 8169 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.196470 8169 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.198601 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.199517 8119 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.202627 8161 tablet_bootstrap.cc:654] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.205727 8009 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.208113 8160 tablet_bootstrap.cc:492] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.208659 8160 ts_tablet_manager.cc:1403] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.006s
I20260616 19:57:53.209002 8169 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.209453 8169 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.209170 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.209803 8169 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.210124 7969 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.211632 8160 raft_consensus.cc:359] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.212387 8160 raft_consensus.cc:385] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.212687 8160 raft_consensus.cc:740] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.213515 8160 consensus_queue.cc:260] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.217855 8169 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.221521 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.222044 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:53.222843 8119 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.222278 7969 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.230512 8159 tablet_bootstrap.cc:492] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.231195 8159 ts_tablet_manager.cc:1403] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.003s
I20260616 19:57:53.234087 7969 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:53.234611 8161 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.234236 8159 raft_consensus.cc:359] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.234962 8159 raft_consensus.cc:385] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.235026 8161 ts_tablet_manager.cc:1403] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.029s
I20260616 19:57:53.235273 8159 raft_consensus.cc:740] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.235361 8011 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.237403 8160 ts_tablet_manager.cc:1434] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.028s user 0.012s sys 0.011s
I20260616 19:57:53.237708 8161 raft_consensus.cc:359] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.238389 8161 raft_consensus.cc:385] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.238504 8160 tablet_bootstrap.cc:492] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.238689 8161 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.239523 8161 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.239976 8119 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:53.241806 8169 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.242669 8169 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.243130 8169 raft_consensus.cc:697] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:53.246035 8160 tablet_bootstrap.cc:654] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.244494 8159 consensus_queue.cc:260] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.245040 8161 ts_tablet_manager.cc:1434] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260616 19:57:53.252266 8161 tablet_bootstrap.cc:492] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.253557 8225 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.254225 8159 ts_tablet_manager.cc:1434] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.023s user 0.008s sys 0.000s
I20260616 19:57:53.254058 8225 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.255167 8159 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.256407 8225 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.257726 8161 tablet_bootstrap.cc:654] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.257936 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.258939 8160 tablet_bootstrap.cc:492] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.259418 8160 ts_tablet_manager.cc:1403] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.006s
I20260616 19:57:53.259706 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.260512 7969 raft_consensus.cc:2393] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:53.262343 8009 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:53.262213 8160 raft_consensus.cc:359] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.263140 8160 raft_consensus.cc:385] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.263466 8160 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.263499 8225 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.264382 8160 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.268836 8161 tablet_bootstrap.cc:492] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.269471 8161 ts_tablet_manager.cc:1403] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20260616 19:57:53.270774 8159 tablet_bootstrap.cc:654] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.272845 8225 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.272708 8161 raft_consensus.cc:359] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.273417 8161 raft_consensus.cc:385] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.273705 8161 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.274475 8161 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.275496 8169 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.276489 8160 ts_tablet_manager.cc:1434] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.017s user 0.006s sys 0.009s
I20260616 19:57:53.281962 8160 tablet_bootstrap.cc:492] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.284981 8159 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.285538 8159 ts_tablet_manager.cc:1403] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.007s
I20260616 19:57:53.285871 8161 ts_tablet_manager.cc:1434] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260616 19:57:53.288720 8160 tablet_bootstrap.cc:654] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.288307 8159 raft_consensus.cc:359] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.289177 8159 raft_consensus.cc:385] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.289502 8159 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.290427 8159 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.291366 8161 tablet_bootstrap.cc:492] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.291188 7868 catalog_manager.cc:5696] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:53.297972 8159 ts_tablet_manager.cc:1434] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.012s user 0.003s sys 0.004s
I20260616 19:57:53.299015 8159 tablet_bootstrap.cc:492] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.300834 8160 tablet_bootstrap.cc:492] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.301397 8160 ts_tablet_manager.cc:1403] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.003s
I20260616 19:57:53.305771 8160 raft_consensus.cc:359] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.306500 8160 raft_consensus.cc:385] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.306669 8159 tablet_bootstrap.cc:654] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.306802 8160 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.307284 8161 tablet_bootstrap.cc:654] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.307860 8160 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.310225 8160 ts_tablet_manager.cc:1434] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260616 19:57:53.311163 8160 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.316609 8159 tablet_bootstrap.cc:492] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.317190 8159 ts_tablet_manager.cc:1403] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.005s
I20260616 19:57:53.317662 8160 tablet_bootstrap.cc:654] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.321573 8161 tablet_bootstrap.cc:492] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.321470 8159 raft_consensus.cc:359] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.322065 8161 ts_tablet_manager.cc:1403] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.007s
I20260616 19:57:53.322247 8159 raft_consensus.cc:385] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.322537 8159 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.323660 8159 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.325717 8161 raft_consensus.cc:359] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.326387 8161 raft_consensus.cc:385] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.326658 8161 raft_consensus.cc:740] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.327312 8161 consensus_queue.cc:260] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.329097 8159 ts_tablet_manager.cc:1434] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260616 19:57:53.329201 8161 ts_tablet_manager.cc:1434] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260616 19:57:53.330109 8159 tablet_bootstrap.cc:492] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.330067 8199 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.330614 8199 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.332619 8161 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.333763 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.334414 7969 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.335667 8085 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:53.336525 8208 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.336866 8208 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.337177 8208 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.338609 8161 tablet_bootstrap.cc:654] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.344015 8161 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.344396 8159 tablet_bootstrap.cc:654] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.344440 8161 ts_tablet_manager.cc:1403] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260616 19:57:53.345431 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.346064 8045 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.347270 8161 raft_consensus.cc:359] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.347913 8161 raft_consensus.cc:385] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.348196 8161 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.332613 8199 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.348806 8161 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.350536 8161 ts_tablet_manager.cc:1434] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260616 19:57:53.351248 8161 tablet_bootstrap.cc:492] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.350965 8208 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.358045 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.358647 7969 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.358889 8161 tablet_bootstrap.cc:654] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.359982 8166 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:53.366255 7969 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.367497 8085 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:53.368358 8166 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.368844 8166 raft_consensus.cc:697] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:53.369656 8166 consensus_queue.cc:237] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.353643 8160 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.373330 8160 ts_tablet_manager.cc:1403] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.003s
I20260616 19:57:53.376202 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:53.377337 8044 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.385608 8044 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.386413 8208 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.376029 8160 raft_consensus.cc:359] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.393316 8160 raft_consensus.cc:385] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.393677 8200 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.393644 8160 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.394814 8160 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.396478 8160 ts_tablet_manager.cc:1434] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260616 19:57:53.397840 8160 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:53.394220 8200 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.403295 8186 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.403789 8186 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.404701 8159 tablet_bootstrap.cc:492] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.405143 8159 ts_tablet_manager.cc:1403] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.075s user 0.023s sys 0.033s
I20260616 19:57:53.409178 8186 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.410864 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.411180 8161 tablet_bootstrap.cc:492] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.411569 8119 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:53.411711 8161 ts_tablet_manager.cc:1403] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.061s user 0.016s sys 0.035s
I20260616 19:57:53.412601 8169 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.413053 8169 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.413579 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.414182 7969 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.414713 8161 raft_consensus.cc:359] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.415382 8161 raft_consensus.cc:385] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.415241 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:53.415652 8161 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.416544 8186 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.416488 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.417014 8186 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.417068 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.417505 8186 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.416330 8161 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.417240 8045 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:53.417841 7970 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.419857 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.420429 8161 ts_tablet_manager.cc:1434] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260616 19:57:53.420236 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.421110 8225 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.421285 7934 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:53.421523 8225 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.421676 8161 tablet_bootstrap.cc:492] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.421882 8225 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.429132 8159 raft_consensus.cc:359] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.435712 8159 raft_consensus.cc:385] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.436137 8159 raft_consensus.cc:740] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.437021 8159 consensus_queue.cc:260] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.418264 8169 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.449877 8169 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.450341 8169 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.454068 8169 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.454159 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.449662 8161 tablet_bootstrap.cc:654] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.454777 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.455397 7970 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.455061 8119 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.458498 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.459352 8169 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.424891 8200 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.459827 8169 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.460367 8169 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.460911 8199 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:53.462471 7868 catalog_manager.cc:5696] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
W20260616 19:57:53.478056 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 56850.7us
I20260616 19:57:53.460309 8200 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.441108 8166 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260616 19:57:53.482287 8159 ts_tablet_manager.cc:1434] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.077s user 0.016s sys 0.009s
I20260616 19:57:53.481174 8200 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.482860 8200 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.485884 8159 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.487003 8160 tablet_bootstrap.cc:654] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:53.481145 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 52629.6us
I20260616 19:57:53.499593 8240 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.501427 8240 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.502174 8186 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.503615 8240 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.509019 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.509706 8225 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.510277 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:57:53.510501 7935 leader_election.cc:343] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.510859 8043 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.521754 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.522258 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.522859 7969 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.510964 8186 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.515894 8169 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.525722 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.521384 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.535751 8119 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.542034 8166 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.541880 8241 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.542688 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.542884 8241 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.542562 8166 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.543452 8159 tablet_bootstrap.cc:654] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.578359 7966 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:53.536388 7934 leader_election.cc:343] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.570591 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:53.583516 8119 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.547421 8208 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.546523 8225 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.593921 8225 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.567750 8043 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.559886 7969 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:53.550166 8228 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.587459 7934 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.598829 8240 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.569149 8169 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.591375 8208 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:53.599038 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 64457.3us
I20260616 19:57:53.593324 8241 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.598150 8228 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.602514 8084 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.597563 8225 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.603096 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.603438 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.588250 7967 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.607455 8208 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.609525 8201 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.610224 8119 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:53.610417 8201 raft_consensus.cc:697] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:53.609429 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.611289 8225 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.605733 8166 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.611603 7969 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.611825 8161 tablet_bootstrap.cc:492] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.596917 8200 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.612617 8161 ts_tablet_manager.cc:1403] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.191s user 0.015s sys 0.004s
I20260616 19:57:53.611318 8201 consensus_queue.cc:237] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.615396 8200 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.615705 7968 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:57:53.616425 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:53.617849 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:53.617909 8161 raft_consensus.cc:359] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.618374 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.618654 8161 raft_consensus.cc:385] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.618824 8045 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:53.618968 7968 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.619912 8208 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.620576 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.620899 8208 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.621111 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.621641 8166 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.621753 7969 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.622156 8200 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.622097 8166 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.622501 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:53.622633 8200 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.623045 8117 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.618958 8161 raft_consensus.cc:740] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.621201 8044 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.625335 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.625962 8042 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.626291 8166 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.623874 8247 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.626845 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.627383 8247 raft_consensus.cc:697] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:53.627992 8011 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.639897 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.628873 8161 consensus_queue.cc:260] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.651968 8160 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:53.649924 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.652541 8160 ts_tablet_manager.cc:1403] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.255s user 0.005s sys 0.023s
I20260616 19:57:53.613545 8043 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.639802 8200 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.657240 8161 ts_tablet_manager.cc:1434] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.044s user 0.007s sys 0.000s
I20260616 19:57:53.637315 8225 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.639797 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.658420 8161 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.628523 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.639060 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.623055 8085 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:53.660526 8042 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.661767 7935 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.662484 8117 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:53.663280 8200 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.662200 8208 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.664784 7934 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.628274 8247 consensus_queue.cc:237] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.640838 8241 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260616 19:57:53.656904 8228 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.668047 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.659000 8225 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.665771 7969 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.663750 8118 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:53.671487 8166 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260616 19:57:53.671836 8085 leader_election.cc:343] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.667519 8200 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:57:53.669618 8225 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.671984 8084 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.673544 8011 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.674823 8228 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.674749 8084 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.675343 8241 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.676754 8241 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.676827 8186 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.677140 8241 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.677201 8186 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.677600 8186 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.679674 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.674330 8166 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.675280 8228 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.683357 8160 raft_consensus.cc:359] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.684352 8160 raft_consensus.cc:385] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.684847 8160 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.685491 8009 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:53.685896 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.681527 8166 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.685642 8160 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.673956 8208 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.688213 8248 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.688786 8248 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:53.689658 8248 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.688221 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.691228 7969 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.692381 8208 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.692749 8208 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.681520 8228 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.712124 8160 ts_tablet_manager.cc:1434] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.059s user 0.003s sys 0.014s
I20260616 19:57:53.687842 8159 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.713192 8159 ts_tablet_manager.cc:1403] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.230s user 0.008s sys 0.008s
I20260616 19:57:53.713374 8160 tablet_bootstrap.cc:492] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
W20260616 19:57:53.712605 8084 leader_election.cc:343] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.714743 8241 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.720408 8241 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.720568 8169 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:53.722489 8161 tablet_bootstrap.cc:654] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.722440 7867 catalog_manager.cc:5696] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260616 19:57:53.726661 8085 leader_election.cc:343] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.726588 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.727176 8085 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.727468 7969 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.726972 8186 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.728809 8201 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.731050 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.731665 7966 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.735452 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:57:53.747076 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:53.752743 8201 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.753283 8201 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.734990 8186 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.759850 7868 catalog_manager.cc:5696] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:53.751740 8118 raft_consensus.cc:2393] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:53.789162 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:53.791256 8201 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.792459 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:53.793078 8042 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.793448 8166 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.794389 8159 raft_consensus.cc:359] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.795547 8159 raft_consensus.cc:385] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.795651 8166 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.799002 8159 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.749642 8208 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.797590 7969 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.809634 8159 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:53.800459 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 65651us
I20260616 19:57:53.812852 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:53.813681 7965 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:53.804314 7967 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:53.818195 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:53.822896 8159 ts_tablet_manager.cc:1434] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.109s user 0.006s sys 0.002s
I20260616 19:57:53.823884 8159 tablet_bootstrap.cc:492] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.824328 8228 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.824553 8085 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:53.824936 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:57:53.827019 8251 log.cc:927] Time spent T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.061s user 0.000s sys 0.001s
I20260616 19:57:53.828187 8043 raft_consensus.cc:2393] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:53.831530 7968 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:53.828805 8084 leader_election.cc:343] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.829053 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:53.830511 8248 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:53.836829 8085 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.838033 7966 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.838231 8186 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.838800 8186 raft_consensus.cc:697] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:53.838972 8241 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:53.839316 8241 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:53.839658 8186 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.839658 8241 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.847633 7868 catalog_manager.cc:5696] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:53.849326 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:53.849877 8043 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.846161 8208 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.852257 8085 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.852612 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.853257 8166 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.853767 8166 raft_consensus.cc:697] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:53.854605 8166 consensus_queue.cc:237] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.855093 8225 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.856627 8045 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:53.856735 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.853356 7966 raft_consensus.cc:2393] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:57:53.860311 8085 leader_election.cc:343] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.861157 8241 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.864372 8241 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:53.865206 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.866039 7966 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:53.866712 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:53.846119 8228 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:57:53.877051 8084 leader_election.cc:343] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:53.900779 8159 tablet_bootstrap.cc:654] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.879500 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:53.939544 8009 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:53.939862 8161 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:53.891575 7968 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.929376 7867 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:53.941035 8161 ts_tablet_manager.cc:1403] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.283s user 0.009s sys 0.005s
I20260616 19:57:53.941999 8085 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.942787 8199 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.929936 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:53.943352 8199 raft_consensus.cc:697] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:53.944223 8199 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.945407 8161 raft_consensus.cc:359] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.946012 8161 raft_consensus.cc:385] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.879675 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:53.946331 8161 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.947258 7965 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:53.947063 8161 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.879204 8042 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.949218 8161 ts_tablet_manager.cc:1434] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.008s user 0.003s sys 0.002s
I20260616 19:57:53.950146 8161 tablet_bootstrap.cc:492] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:53.919140 8043 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.877278 8186 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.063s
I20260616 19:57:53.953786 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:53.955768 8166 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.884364 8225 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.956697 8225 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.957633 8166 raft_consensus.cc:697] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:53.958943 8166 consensus_queue.cc:237] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.962508 7966 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:53.964042 8085 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:53.965296 8241 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:53.965791 8241 raft_consensus.cc:697] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:53.963689 8186 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:53.966621 8241 consensus_queue.cc:237] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.968441 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:53.971813 8228 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:53.973453 8160 tablet_bootstrap.cc:654] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.975908 8159 tablet_bootstrap.cc:492] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:53.976465 8159 ts_tablet_manager.cc:1403] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.153s user 0.012s sys 0.029s
I20260616 19:57:53.979161 8159 raft_consensus.cc:359] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.979904 8159 raft_consensus.cc:385] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:53.980248 8159 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:53.981029 8159 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.983505 8159 ts_tablet_manager.cc:1434] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260616 19:57:53.984552 8159 tablet_bootstrap.cc:492] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:53.991809 8228 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:53.992481 8228 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:53.994103 8161 tablet_bootstrap.cc:654] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:53.995091 8228 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:53.997709 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:53.998200 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:53.999656 8009 leader_election.cc:343] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
W20260616 19:57:54.000363 8011 leader_election.cc:343] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.000873 8011 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:54.002102 8228 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:54.002887 8159 tablet_bootstrap.cc:654] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.004417 8200 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.004868 8200 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.007483 8200 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.009352 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.009989 8118 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:54.015000 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:54.017056 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:54.017607 8161 tablet_bootstrap.cc:492] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.018218 8161 ts_tablet_manager.cc:1403] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.068s user 0.014s sys 0.008s
W20260616 19:57:54.018934 7935 leader_election.cc:343] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.021898 8161 raft_consensus.cc:359] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.022823 8161 raft_consensus.cc:385] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.023177 8161 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.024291 8161 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.025926 8243 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:54.025820 7868 catalog_manager.cc:5696] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:54.026439 8243 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:54.026921 8243 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.028103 7868 catalog_manager.cc:5696] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:54.032611 8161 ts_tablet_manager.cc:1434] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.014s user 0.010s sys 0.000s
I20260616 19:57:54.032279 7868 catalog_manager.cc:5696] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:54.034170 8161 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:54.034461 7868 catalog_manager.cc:5696] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:54.049263 8160 tablet_bootstrap.cc:492] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:54.049726 8160 ts_tablet_manager.cc:1403] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.337s user 0.032s sys 0.039s
I20260616 19:57:54.052657 8160 raft_consensus.cc:359] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.053328 8160 raft_consensus.cc:385] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.053604 8160 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.054231 8160 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.054776 8161 tablet_bootstrap.cc:654] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.056696 8160 ts_tablet_manager.cc:1434] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260616 19:57:54.059038 8160 tablet_bootstrap.cc:492] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:54.059077 8243 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.064245 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:57:54.065582 7935 leader_election.cc:343] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.073355 8243 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.073661 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:54.074429 8118 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.078863 8161 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.079320 8161 ts_tablet_manager.cc:1403] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.005s
I20260616 19:57:54.081808 8161 raft_consensus.cc:359] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.082477 8161 raft_consensus.cc:385] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.082746 8161 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.082840 8118 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:54.083420 8161 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.084125 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:54.084538 8159 tablet_bootstrap.cc:492] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:54.084980 8159 ts_tablet_manager.cc:1403] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.101s user 0.020s sys 0.010s
I20260616 19:57:54.085453 8161 ts_tablet_manager.cc:1434] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260616 19:57:54.086472 8161 tablet_bootstrap.cc:492] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:54.087196 8160 tablet_bootstrap.cc:654] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.087512 8159 raft_consensus.cc:359] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.088579 8159 raft_consensus.cc:385] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.110733 8159 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.088666 8243 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:54.094522 8241 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.114701 8241 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.115140 8159 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.117264 8241 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:54.117508 8159 ts_tablet_manager.cc:1434] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.032s user 0.004s sys 0.016s
I20260616 19:57:54.118994 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:54.120957 8161 tablet_bootstrap.cc:654] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:54.120869 8085 leader_election.cc:343] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.121981 8159 tablet_bootstrap.cc:492] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:54.125061 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:54.127336 8084 leader_election.cc:343] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.129101 8160 tablet_bootstrap.cc:492] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:54.128417 8084 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:54.130005 8160 ts_tablet_manager.cc:1403] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.071s user 0.010s sys 0.011s
I20260616 19:57:54.130283 8208 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:54.131536 8159 tablet_bootstrap.cc:654] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.132575 8160 raft_consensus.cc:359] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.133260 8160 raft_consensus.cc:385] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.133517 8160 raft_consensus.cc:740] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.134893 8241 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:54.134385 8160 consensus_queue.cc:260] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.141711 8160 ts_tablet_manager.cc:1434] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260616 19:57:54.142915 8160 tablet_bootstrap.cc:492] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:54.112782 8243 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:54.144975 8243 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.151736 8228 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.152464 8160 tablet_bootstrap.cc:654] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.158349 8160 tablet_bootstrap.cc:492] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:54.158823 8160 ts_tablet_manager.cc:1403] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20260616 19:57:54.161243 8166 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.161736 8166 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.163177 8208 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.163321 8228 consensus_queue.cc:1048] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.163789 8166 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:54.164687 8159 tablet_bootstrap.cc:492] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:54.165134 8159 ts_tablet_manager.cc:1403] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.043s user 0.016s sys 0.011s
I20260616 19:57:54.166524 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:54.168612 8085 leader_election.cc:343] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.170352 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:54.168644 8159 raft_consensus.cc:359] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:54.172053 8084 leader_election.cc:343] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.172691 8084 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:54.173668 8159 raft_consensus.cc:385] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.174083 8159 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.174270 8208 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:54.176106 8159 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.180573 8159 ts_tablet_manager.cc:1434] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.015s user 0.009s sys 0.000s
I20260616 19:57:54.181715 8159 tablet_bootstrap.cc:492] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:54.185017 8160 raft_consensus.cc:359] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.186028 8160 raft_consensus.cc:385] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.186309 8160 raft_consensus.cc:740] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.192907 8161 tablet_bootstrap.cc:492] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.193465 8161 ts_tablet_manager.cc:1403] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.107s user 0.025s sys 0.031s
I20260616 19:57:54.194427 7867 catalog_manager.cc:5696] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:54.196868 8161 raft_consensus.cc:359] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.197841 8161 raft_consensus.cc:385] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.198189 8161 raft_consensus.cc:740] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.199087 8161 consensus_queue.cc:260] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.199849 8278 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.200444 8278 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.201458 8161 ts_tablet_manager.cc:1434] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260616 19:57:54.202456 8161 tablet_bootstrap.cc:492] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:54.203994 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:54.204913 8043 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:54.204785 8160 consensus_queue.cc:260] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.204041 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.209736 8161 tablet_bootstrap.cc:654] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.214649 8159 tablet_bootstrap.cc:654] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.215029 7935 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:54.217010 8160 ts_tablet_manager.cc:1434] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.058s user 0.007s sys 0.002s
I20260616 19:57:54.218001 8160 tablet_bootstrap.cc:492] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:54.221866 8278 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.223974 8243 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.230862 8248 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.233557 8248 consensus_queue.cc:1048] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:54.240195 8160 tablet_bootstrap.cc:654] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.244941 8159 tablet_bootstrap.cc:492] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:54.245592 8159 ts_tablet_manager.cc:1403] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.064s user 0.017s sys 0.000s
I20260616 19:57:54.248981 8159 raft_consensus.cc:359] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.249704 8159 raft_consensus.cc:385] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.249974 8159 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.252652 8243 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:54.263635 8159 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.266103 8159 ts_tablet_manager.cc:1434] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260616 19:57:54.267112 8159 tablet_bootstrap.cc:492] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:54.268297 8161 tablet_bootstrap.cc:492] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.268832 8161 ts_tablet_manager.cc:1403] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.067s user 0.031s sys 0.017s
I20260616 19:57:54.271785 8161 raft_consensus.cc:359] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.272562 8161 raft_consensus.cc:385] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.272826 8161 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.278541 8161 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.280807 8159 tablet_bootstrap.cc:654] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.280858 8161 ts_tablet_manager.cc:1434] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260616 19:57:54.281909 8161 tablet_bootstrap.cc:492] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:54.285729 8160 tablet_bootstrap.cc:492] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:54.286312 8160 ts_tablet_manager.cc:1403] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.069s user 0.007s sys 0.016s
I20260616 19:57:54.288291 8161 tablet_bootstrap.cc:654] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.289050 8160 raft_consensus.cc:359] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.289778 8160 raft_consensus.cc:385] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.290067 8160 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.290726 8160 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.297748 8160 ts_tablet_manager.cc:1434] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260616 19:57:54.298496 8159 tablet_bootstrap.cc:492] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:54.298902 8160 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:54.299005 8159 ts_tablet_manager.cc:1403] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.000s
I20260616 19:57:54.301744 8161 tablet_bootstrap.cc:492] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.302888 8161 ts_tablet_manager.cc:1403] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.000s
I20260616 19:57:54.304782 8159 raft_consensus.cc:359] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.305096 8161 raft_consensus.cc:359] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.305442 8160 tablet_bootstrap.cc:654] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.305594 8159 raft_consensus.cc:385] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.305720 8161 raft_consensus.cc:385] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.306273 8159 raft_consensus.cc:740] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.306412 8161 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.307147 8161 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.309271 8161 ts_tablet_manager.cc:1434] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260616 19:57:54.307299 8159 consensus_queue.cc:260] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.311154 8159 ts_tablet_manager.cc:1434] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260616 19:57:54.312371 8159 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:54.314270 8160 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:54.314718 8160 ts_tablet_manager.cc:1403] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260616 19:57:54.317204 8160 raft_consensus.cc:359] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.317711 8160 raft_consensus.cc:385] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.318001 8160 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.318077 8228 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.318652 8159 tablet_bootstrap.cc:654] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.318589 8228 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.318694 8160 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.320206 8248 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.320995 8160 ts_tablet_manager.cc:1434] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260616 19:57:54.320641 8248 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.312386 8161 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:54.322600 8228 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.322710 8248 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.323267 8160 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:54.324924 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.325604 8116 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:54.327041 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.328145 8161 tablet_bootstrap.cc:654] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.328213 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:54.329336 7966 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:54.329344 8009 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:54.331585 8011 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:54.332589 8160 tablet_bootstrap.cc:654] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.333838 8248 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:54.334237 8248 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:54.334594 8248 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.336431 8228 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.348254 8161 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.354717 8161 ts_tablet_manager.cc:1403] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.019s
I20260616 19:57:54.329319 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:54.358553 8228 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:54.355938 7968 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:54.358886 8248 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.360937 8166 consensus_queue.cc:1048] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.362505 8228 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.362867 8228 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.364567 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:54.364820 8228 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.365173 8118 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.365532 8161 raft_consensus.cc:359] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.366067 8161 raft_consensus.cc:385] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.366230 8161 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.366768 8161 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.367523 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:54.368173 7966 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:54.369166 8011 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:54.369819 8161 ts_tablet_manager.cc:1434] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.015s user 0.002s sys 0.005s
I20260616 19:57:54.370438 8159 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:54.370702 8161 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:54.370842 8159 ts_tablet_manager.cc:1403] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.059s user 0.012s sys 0.009s
I20260616 19:57:54.373257 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.373842 8225 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:54.373885 8116 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:54.373435 8159 raft_consensus.cc:359] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.374346 8225 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:54.374580 8159 raft_consensus.cc:385] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.374718 8225 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.374927 8159 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.375684 8159 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.376705 8161 tablet_bootstrap.cc:654] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.377776 8159 ts_tablet_manager.cc:1434] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260616 19:57:54.383702 8241 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.384977 8161 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.385424 8161 ts_tablet_manager.cc:1403] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260616 19:57:54.386281 8159 tablet_bootstrap.cc:492] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:54.388645 8161 raft_consensus.cc:359] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.389518 8161 raft_consensus.cc:385] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.389861 8161 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.390537 8161 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.392349 8161 ts_tablet_manager.cc:1434] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260616 19:57:54.393249 8161 tablet_bootstrap.cc:492] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:54.393291 8159 tablet_bootstrap.cc:654] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.396333 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:54.396971 7968 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.399088 8241 consensus_queue.cc:1048] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:54.400547 8243 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.401248 8243 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.383664 8225 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.404618 8243 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.405714 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:54.406347 8042 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:54.409013 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:54.409459 7935 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:54.409654 7965 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.410598 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.411233 8116 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:54.412997 8243 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:54.413455 8243 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:54.413576 7968 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:54.413904 8243 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.414778 8011 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:54.415479 8225 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.415619 8228 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:54.415512 8118 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:54.415699 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:54.416807 8228 raft_consensus.cc:697] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:54.417490 8116 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.417968 8228 consensus_queue.cc:237] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.419164 8248 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.400784 8161 tablet_bootstrap.cc:654] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.429550 8166 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.433610 8160 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:54.434262 8160 ts_tablet_manager.cc:1403] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.111s user 0.021s sys 0.001s
I20260616 19:57:54.438397 8160 raft_consensus.cc:359] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.444952 8160 raft_consensus.cc:385] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.445222 8160 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.445856 8160 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.450233 7965 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:54.452788 8160 ts_tablet_manager.cc:1434] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.018s user 0.007s sys 0.001s
I20260616 19:57:54.453436 8011 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:54.453751 8160 tablet_bootstrap.cc:492] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:54.460376 8169 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:54.463699 8208 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:54.465253 8278 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.465785 8278 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.467903 8278 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.468744 8199 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:54.470077 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:54.471761 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.472414 8118 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:54.474166 7934 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:54.474932 8278 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:54.475342 8278 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:54.476980 8278 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.459929 8116 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
W20260616 19:57:54.474924 7935 leader_election.cc:343] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.495952 8166 consensus_queue.cc:1048] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260616 19:57:54.534216 8159 tablet_bootstrap.cc:492] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:54.534741 8159 ts_tablet_manager.cc:1403] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.149s user 0.047s sys 0.043s
I20260616 19:57:54.524575 8169 raft_consensus.cc:697] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:54.535903 8243 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.549784 8243 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.550901 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:54.551543 8116 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.551287 8278 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.554394 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:54.555047 8041 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.565124 8160 tablet_bootstrap.cc:654] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:54.566994 8169 consensus_queue.cc:237] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.569841 8248 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.572805 8248 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.564338 8241 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260616 19:57:54.668797 8161 tablet_bootstrap.cc:492] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:54.707337 8161 ts_tablet_manager.cc:1403] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.314s user 0.026s sys 0.006s
I20260616 19:57:54.707559 8159 raft_consensus.cc:359] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.709627 8159 raft_consensus.cc:385] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:54.709978 8159 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:54.686852 8228 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.686057 8248 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.689640 8199 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.717497 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:54.720319 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:54.727128 8117 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.722024 8243 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260616 19:57:54.741335 8159 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.716866 8278 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.718544 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:54.721350 8228 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.737236 8199 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.762439 8159 ts_tablet_manager.cc:1434] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.227s user 0.007s sys 0.023s
I20260616 19:57:54.765208 8159 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:54.770092 8304 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:57:54.761132 8011 leader_election.cc:343] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
W20260616 19:57:54.776579 7935 leader_election.cc:343] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:54.723726 7866 catalog_manager.cc:5696] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:54.799153 8117 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:54.842207 8298 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.842926 8298 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.850545 8044 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:54.851445 8298 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.851961 8228 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.844420 7934 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:54.853181 8278 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:54.853631 8278 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.854171 8199 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:54.865476 8278 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:54.866780 8116 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:54.867197 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:54.867818 8117 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:54.790045 8304 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:54.887163 7934 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:54.868404 8243 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:54.888186 8278 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:54.889278 7934 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:55.001997 8161 raft_consensus.cc:359] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.003540 8161 raft_consensus.cc:385] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:55.003904 8161 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:55.004750 8161 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:55.005517 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:54.930801 8041 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:55.015476 8161 ts_tablet_manager.cc:1434] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.307s user 0.039s sys 0.020s
I20260616 19:57:55.016446 8161 tablet_bootstrap.cc:492] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:55.008327 8278 raft_consensus.cc:697] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:55.008510 8243 raft_consensus.cc:697] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:55.013011 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:55.054185 8159 tablet_bootstrap.cc:654] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:55.057984 8319 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:55.059501 8304 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:55.049715 7967 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:55.071623 8160 tablet_bootstrap.cc:492] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:55.072137 8160 ts_tablet_manager.cc:1403] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.619s user 0.036s sys 0.016s
W20260616 19:57:55.035594 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 146388us
I20260616 19:57:55.068635 8319 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:55.080099 8318 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260616 19:57:55.081859 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:55.076536 8199 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260616 19:57:55.094112 8319 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:55.105230 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:55.098511 8045 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:55.105136 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:55.096249 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 77675.6us
I20260616 19:57:55.107723 8039 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:55.094496 8278 consensus_queue.cc:237] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.107636 8038 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:55.114624 8160 raft_consensus.cc:359] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.115432 8160 raft_consensus.cc:385] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:55.115134 8161 tablet_bootstrap.cc:654] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:55.115741 8160 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:55.098863 8243 consensus_queue.cc:237] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.116487 8160 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.133639 8160 ts_tablet_manager.cc:1434] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.061s user 0.003s sys 0.003s
I20260616 19:57:55.134447 8160 tablet_bootstrap.cc:492] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:55.126284 8011 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:55.168533 8322 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:55.169458 8322 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:55.169836 8322 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:55.244302 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:55.249743 8085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s user 0.009s sys 0.026s
W20260616 19:57:55.250166 8085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s user 0.009s sys 0.026s
W20260616 19:57:55.250512 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 660659us
W20260616 19:57:55.283228 8320 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.185s user 0.004s sys 0.000s
I20260616 19:57:55.233713 8228 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260616 19:57:55.307245 8084 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:55.317490 8303 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:55.317930 8303 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:55.318313 8303 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:55.323659 8199 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:55.357857 8159 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:55.358527 8159 ts_tablet_manager.cc:1403] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.594s user 0.023s sys 0.012s
W20260616 19:57:55.367763 8225 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s user 0.016s sys 0.012s
W20260616 19:57:55.376965 8301 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.178s user 0.011s sys 0.003s
I20260616 19:57:55.365550 8199 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.378841 8199 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:57:55.374033 8225 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s user 0.017s sys 0.012s
I20260616 19:57:55.385659 8159 raft_consensus.cc:359] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.386476 8159 raft_consensus.cc:385] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:55.386759 8159 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:55.387461 8159 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.388608 8303 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.389521 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:55.394364 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:55.396358 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:55.397225 8118 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:55.416639 8159 ts_tablet_manager.cc:1434] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.058s user 0.007s sys 0.000s
I20260616 19:57:55.397575 8298 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:55.410668 8199 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:55.426044 8161 tablet_bootstrap.cc:492] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:55.426450 8161 ts_tablet_manager.cc:1403] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.410s user 0.035s sys 0.086s
I20260616 19:57:55.418658 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:55.418450 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:55.432226 8322 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.433993 8160 tablet_bootstrap.cc:654] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:55.425299 8199 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:55.444865 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:55.445703 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:55.445968 8303 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:55.448330 8159 tablet_bootstrap.cc:492] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:55.437119 7968 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:55.437300 8199 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:55.439718 8291 log.cc:927] Time spent T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.229s user 0.008s sys 0.007s
I20260616 19:57:55.445709 8041 raft_consensus.cc:2393] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:55.454281 8009 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:55.456817 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:55.440027 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:55.446386 8036 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:55.463400 8228 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:55.474195 8228 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:55.475317 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:55.451133 8319 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:55.458515 8085 leader_election.cc:343] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:55.500564 8159 tablet_bootstrap.cc:654] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:55.467566 8118 raft_consensus.cc:2393] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:55.505669 8161 raft_consensus.cc:359] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.508845 8084 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:55.511893 8161 raft_consensus.cc:385] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:55.512880 8161 raft_consensus.cc:740] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:55.513612 8161 consensus_queue.cc:260] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.509459 8228 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:55.522495 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 64439.6us
I20260616 19:57:55.512084 7964 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:55.522697 8199 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.525347 8161 ts_tablet_manager.cc:1434] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.099s user 0.004s sys 0.003s
I20260616 19:57:55.512691 8303 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:55.532861 8304 consensus_queue.cc:1048] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:55.536585 8319 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:55.537310 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:55.537928 8036 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:55.533675 8085 leader_election.cc:343] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:55.540107 8161 tablet_bootstrap.cc:492] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:55.572482 8243 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:55.534567 8303 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:57:55.606695 8336 log.cc:927] Time spent T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260616 19:57:55.594234 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:55.618423 8303 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:55.627334 8160 tablet_bootstrap.cc:492] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:55.629165 8160 ts_tablet_manager.cc:1403] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.495s user 0.008s sys 0.021s
I20260616 19:57:55.620035 8243 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.629877 8116 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:55.641708 8338 log.cc:927] Time spent T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260616 19:57:55.647097 8160 raft_consensus.cc:359] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.648111 8160 raft_consensus.cc:385] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:55.650763 8160 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:55.656901 8161 tablet_bootstrap.cc:654] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:55.659385 8311 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:55.660120 8311 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:55.670534 7935 consensus_peers.cc:597] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Couldn't send request to peer de4c5ce3ea8345ae9068f825b8d60e19. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:57:55.676990 8323 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260616 19:57:55.679250 8160 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.685904 8160 ts_tablet_manager.cc:1434] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.056s user 0.002s sys 0.006s
I20260616 19:57:55.686895 8160 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:55.680024 8199 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:55.687841 8322 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:55.695205 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:55.689162 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:55.701440 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:55.702188 8036 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:55.703547 7964 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:55.696439 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:55.705250 8243 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:55.699921 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:55.706385 8228 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:55.707248 8085 leader_election.cc:343] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:55.707638 8044 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:55.709651 8085 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:57:55.711308 8084 leader_election.cc:343] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:55.712185 8228 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:55.716480 8159 tablet_bootstrap.cc:492] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:55.717089 8159 ts_tablet_manager.cc:1403] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.269s user 0.024s sys 0.047s
I20260616 19:57:55.721002 8199 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:55.714267 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:55.713583 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:55.712487 8303 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.713927 7935 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:55.729655 8160 tablet_bootstrap.cc:654] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:55.730804 8011 leader_election.cc:343] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
W20260616 19:57:55.731717 7935 leader_election.cc:343] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:55.733152 8319 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:55.733639 8319 raft_consensus.cc:697] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:55.734475 8319 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.728068 8199 raft_consensus.cc:697] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:55.737069 8199 consensus_queue.cc:237] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.741043 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 47) took 1199 ms. Trace:
I20260616 19:57:55.745378 8116 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:55.747115 8009 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:55.747627 8303 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:57:55.750883 8085 leader_election.cc:343] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:55.752790 8322 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:55.756521 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:55.757413 8114 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:55.752786 8248 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260616 19:57:55.758502 8322 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:55.759219 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:55.759810 8040 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:55.760332 7934 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:55.769405 8318 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:55.769944 8318 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:55.770367 8318 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:55.770833 8311 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:55.786319 8161 tablet_bootstrap.cc:492] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:55.786814 8161 ts_tablet_manager.cc:1403] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.247s user 0.013s sys 0.017s
I20260616 19:57:55.780565 8040 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:55.771469 8085 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:55.794529 8084 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:57:55.796965 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 48248.5us
I20260616 19:57:55.797991 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:55.798796 8199 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:55.800226 8304 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:55.803254 8309 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:55.803745 8309 raft_consensus.cc:697] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:55.802644 8161 raft_consensus.cc:359] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.804560 8309 consensus_queue.cc:237] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.805004 8161 raft_consensus.cc:385] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:55.805433 8161 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:55.806291 8161 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.808712 8161 ts_tablet_manager.cc:1434] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.022s user 0.004s sys 0.002s
I20260616 19:57:55.808401 8278 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:55.809783 8161 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:55.741670 7866 rpcz_store.cc:276] 0616 19:57:54.545675 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:57:54.545818 (+ 143us) service_pool.cc:224] Handling call
0616 19:57:55.740957 (+1195139us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:57:55.077975 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:57:55.110124 (+ 32149us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:57:55.110170 (+ 46us) write_op.cc:435] Acquired schema lock
0616 19:57:55.110191 (+ 21us) tablet.cc:662] Decoding operations
0616 19:57:55.110554 (+ 363us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:57:55.110608 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0616 19:57:55.110639 (+ 31us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:57:55.110840 (+ 201us) tablet.cc:701] Row locks acquired
0616 19:57:55.110858 (+ 18us) write_op.cc:260] PREPARE: finished
0616 19:57:55.110988 (+ 130us) write_op.cc:270] Start()
0616 19:57:55.111110 (+ 122us) write_op.cc:276] Timestamp: P: 1781639875110951 usec, L: 0
0616 19:57:55.111131 (+ 21us) op_driver.cc:348] REPLICATION: starting
0616 19:57:55.125485 (+ 14354us) log.cc:844] Serialized 540 byte log entry
0616 19:57:55.384462 (+258977us) op_driver.cc:464] REPLICATION: finished
0616 19:57:55.472234 (+ 87772us) write_op.cc:301] APPLY: starting
0616 19:57:55.494103 (+ 21869us) tablet.cc:1367] starting BulkCheckPresence
0616 19:57:55.503301 (+ 9198us) tablet.cc:1370] finished BulkCheckPresence
0616 19:57:55.504715 (+ 1414us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:57:55.505008 (+ 293us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:57:55.505147 (+ 139us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:57:55.505187 (+ 40us) write_op.cc:312] APPLY: finished
0616 19:57:55.505385 (+ 198us) log.cc:844] Serialized 22 byte log entry
0616 19:57:55.506119 (+ 734us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:57:55.506197 (+ 78us) write_op.cc:454] Released schema lock
0616 19:57:55.506233 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0616 19:57:55.506506 (+ 273us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:57:55.506539 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":71930,"apply.run_cpu_time_us":1661,"apply.run_wall_time_us":47016,"num_ops":1,"prepare.queue_time_us":230072,"prepare.run_cpu_time_us":3486,"prepare.run_wall_time_us":74964,"replication_time_us":271218,"thread_start_us":70530,"threads_started":2}]]}
I20260616 19:57:55.816526 8159 raft_consensus.cc:359] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.819572 8159 raft_consensus.cc:385] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:55.819926 8159 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:55.818506 7865 catalog_manager.cc:5696] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:57:55.802644 8318 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.829149 8159 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.825811 8278 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260616 19:57:55.849802 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:55.850100 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:55.850904 8118 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:55.861106 8009 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:55.862090 8322 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:55.825006 7865 catalog_manager.cc:5696] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:55.865803 7865 catalog_manager.cc:5696] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:55.867722 8322 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:55.868326 8322 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:55.911517 8159 ts_tablet_manager.cc:1434] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.194s user 0.008s sys 0.001s
I20260616 19:57:55.921918 8159 tablet_bootstrap.cc:492] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:55.922346 8009 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:55.935554 8248 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:55.961812 8160 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:55.962435 8160 ts_tablet_manager.cc:1403] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.276s user 0.023s sys 0.028s
I20260616 19:57:55.997789 8319 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.018975 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:56.019904 8319 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:55.998080 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 161411us
I20260616 19:57:56.028542 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.033075 8161 tablet_bootstrap.cc:654] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.014869 8160 raft_consensus.cc:359] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.034957 8160 raft_consensus.cc:385] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.035315 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.033461 8322 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.035315 8160 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.040776 8160 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.044622 8159 tablet_bootstrap.cc:654] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.045300 8160 ts_tablet_manager.cc:1434] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.083s user 0.007s sys 0.011s
I20260616 19:57:56.046026 8160 tablet_bootstrap.cc:492] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
W20260616 19:57:56.012265 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 96896.2us
I20260616 19:57:56.032192 8319 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.038209 7963 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.076610 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.068272 8318 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.091156 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.093406 8043 raft_consensus.cc:2393] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:56.095106 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:56.086179 8115 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.097797 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.098169 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:56.098786 8043 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.103229 8160 tablet_bootstrap.cc:654] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.112143 8009 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:56.114439 8248 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260616 19:57:56.118054 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:56.127857 8115 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:56.138571 7934 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:56.139453 8243 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:56.139948 8243 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:56.140770 8243 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.144873 8043 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:56.146551 8159 tablet_bootstrap.cc:492] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.146994 8159 ts_tablet_manager.cc:1403] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.234s user 0.015s sys 0.000s
I20260616 19:57:56.147985 8161 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:56.148458 8161 ts_tablet_manager.cc:1403] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.339s user 0.005s sys 0.018s
I20260616 19:57:56.151268 8159 raft_consensus.cc:359] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.152056 8159 raft_consensus.cc:385] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.152812 8159 raft_consensus.cc:740] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.154516 8159 consensus_queue.cc:260] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.156104 8161 raft_consensus.cc:359] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.156905 8161 raft_consensus.cc:385] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.157299 8161 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.157004 8159 ts_tablet_manager.cc:1434] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260616 19:57:56.158560 8159 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:56.159116 8161 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.160575 8160 tablet_bootstrap.cc:492] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:56.161003 8160 ts_tablet_manager.cc:1403] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.115s user 0.014s sys 0.009s
I20260616 19:57:56.161553 8161 ts_tablet_manager.cc:1434] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.013s user 0.002s sys 0.005s
I20260616 19:57:56.162065 7963 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:56.162418 8161 tablet_bootstrap.cc:492] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:56.162614 8318 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:56.163501 8160 raft_consensus.cc:359] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.164125 8160 raft_consensus.cc:385] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.164431 8160 raft_consensus.cc:740] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.165189 8160 consensus_queue.cc:260] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.167392 8160 ts_tablet_manager.cc:1434] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260616 19:57:56.168725 8160 tablet_bootstrap.cc:492] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:56.169191 8161 tablet_bootstrap.cc:654] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.175475 8160 tablet_bootstrap.cc:654] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.176970 7867 catalog_manager.cc:5696] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:56.181011 8159 tablet_bootstrap.cc:654] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.186486 8161 tablet_bootstrap.cc:492] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:56.186815 8160 tablet_bootstrap.cc:492] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:56.186926 8161 ts_tablet_manager.cc:1403] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.015s
I20260616 19:57:56.187235 8160 ts_tablet_manager.cc:1403] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260616 19:57:56.190056 8161 raft_consensus.cc:359] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.190213 8160 raft_consensus.cc:359] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.190754 8161 raft_consensus.cc:385] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.190789 8160 raft_consensus.cc:385] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.196302 8160 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.191169 8161 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.192098 8304 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.197495 8304 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.202374 8304 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:56.204617 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.204725 8160 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.201776 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.207692 8160 ts_tablet_manager.cc:1434] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
W20260616 19:57:56.208554 8085 leader_election.cc:343] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.203864 8161 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.212189 8161 ts_tablet_manager.cc:1434] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.025s user 0.008s sys 0.000s
W20260616 19:57:56.213006 8084 leader_election.cc:343] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.213464 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:56.214234 8304 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:56.216215 8160 tablet_bootstrap.cc:492] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:56.216719 8318 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:56.223517 8160 tablet_bootstrap.cc:654] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.227367 8304 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:56.230602 8043 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.234928 8160 tablet_bootstrap.cc:492] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:56.235441 8160 ts_tablet_manager.cc:1403] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.015s
I20260616 19:57:56.238389 8160 raft_consensus.cc:359] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.213575 8161 tablet_bootstrap.cc:492] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:56.239123 8160 raft_consensus.cc:385] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.239444 8160 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.240290 8160 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.243527 8160 ts_tablet_manager.cc:1434] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260616 19:57:56.244833 8304 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260616 19:57:56.256713 8319 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:56.257202 8160 tablet_bootstrap.cc:492] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:56.259729 8159 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.260237 8159 ts_tablet_manager.cc:1403] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.102s user 0.028s sys 0.002s
I20260616 19:57:56.272580 8304 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.273123 8304 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.269703 7865 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:56.283174 8161 tablet_bootstrap.cc:654] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.282524 8306 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:56.285746 8159 raft_consensus.cc:359] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.286578 8159 raft_consensus.cc:385] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.286876 8159 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.291050 8159 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.296224 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.296635 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.298938 8159 ts_tablet_manager.cc:1434] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.038s user 0.004s sys 0.002s
W20260616 19:57:56.299234 8084 leader_election.cc:343] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.298746 7865 catalog_manager.cc:5696] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:56.300017 8159 tablet_bootstrap.cc:492] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
W20260616 19:57:56.304395 8085 leader_election.cc:343] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.304888 8085 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:56.307380 8166 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:56.308192 8160 tablet_bootstrap.cc:654] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.295591 8304 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:56.318493 8304 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.319133 8304 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.324249 8319 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.324771 8319 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.328238 8319 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.336766 8159 tablet_bootstrap.cc:654] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.337522 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.339341 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.342121 8166 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260616 19:57:56.344486 8115 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:56.347345 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.344837 8304 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:56.349941 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.361763 8043 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
W20260616 19:57:56.364501 8084 leader_election.cc:343] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.365617 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.366539 8319 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.366904 8319 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.367237 8319 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.373140 8160 tablet_bootstrap.cc:492] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
W20260616 19:57:56.373366 8085 leader_election.cc:343] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.373777 8160 ts_tablet_manager.cc:1403] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.117s user 0.033s sys 0.036s
I20260616 19:57:56.374028 8085 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:56.374996 8166 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:56.379949 7867 catalog_manager.cc:5696] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:56.393409 8319 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.386394 8160 raft_consensus.cc:359] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.396070 8160 raft_consensus.cc:385] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.396864 8160 raft_consensus.cc:740] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.419003 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:56.419835 8043 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.419353 8160 consensus_queue.cc:260] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.422196 8160 ts_tablet_manager.cc:1434] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.048s user 0.008s sys 0.000s
I20260616 19:57:56.423621 8160 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:56.426883 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.427543 8115 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.430083 8161 tablet_bootstrap.cc:492] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:56.430594 8161 ts_tablet_manager.cc:1403] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.217s user 0.022s sys 0.029s
I20260616 19:57:56.428603 8319 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.435496 8161 raft_consensus.cc:359] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.436225 8161 raft_consensus.cc:385] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.436501 8161 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.437258 8161 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.441390 8161 ts_tablet_manager.cc:1434] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260616 19:57:56.441515 8159 tablet_bootstrap.cc:492] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.441943 8159 ts_tablet_manager.cc:1403] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.142s user 0.021s sys 0.028s
I20260616 19:57:56.442423 8161 tablet_bootstrap.cc:492] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:56.445572 8043 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:56.446776 8159 raft_consensus.cc:359] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.447492 8159 raft_consensus.cc:385] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.447774 8159 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.447765 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.448866 8354 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.448714 8159 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.449357 8354 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.450043 8318 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:56.450984 8159 ts_tablet_manager.cc:1434] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260616 19:57:56.451241 8318 raft_consensus.cc:697] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:56.452054 8318 consensus_queue.cc:237] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.455600 8159 tablet_bootstrap.cc:492] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:56.455992 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.456827 8043 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:56.458642 8160 tablet_bootstrap.cc:654] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.458114 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.462945 8118 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:56.453377 8115 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:56.460249 8354 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.478257 7935 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:56.480701 8354 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.482964 8161 tablet_bootstrap.cc:654] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.497535 8159 tablet_bootstrap.cc:654] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.508905 7867 catalog_manager.cc:5696] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:56.513648 8354 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:56.518199 8160 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:56.518680 8160 ts_tablet_manager.cc:1403] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.095s user 0.014s sys 0.009s
I20260616 19:57:56.522251 8160 raft_consensus.cc:359] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.523046 8160 raft_consensus.cc:385] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.523414 8160 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.524219 8160 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.526585 8160 ts_tablet_manager.cc:1434] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260616 19:57:56.529042 8160 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:56.534200 8161 tablet_bootstrap.cc:492] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:56.534693 8161 ts_tablet_manager.cc:1403] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.092s user 0.007s sys 0.005s
I20260616 19:57:56.537622 8322 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.537894 8159 tablet_bootstrap.cc:492] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.538341 8159 ts_tablet_manager.cc:1403] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.083s user 0.010s sys 0.008s
I20260616 19:57:56.538183 8322 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.539258 8160 tablet_bootstrap.cc:654] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.541330 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.541118 8159 raft_consensus.cc:359] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.541833 8159 raft_consensus.cc:385] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.542112 8159 raft_consensus.cc:740] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
W20260616 19:57:56.542968 8011 leader_election.cc:343] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.543028 8159 consensus_queue.cc:260] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.545257 8159 ts_tablet_manager.cc:1434] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260616 19:57:56.546568 8159 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:56.552903 8159 tablet_bootstrap.cc:654] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.556738 8322 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.557958 8159 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.557864 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.558501 8159 ts_tablet_manager.cc:1403] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
W20260616 19:57:56.559789 8009 leader_election.cc:343] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.560460 8009 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:56.561496 8159 raft_consensus.cc:359] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.561928 8322 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:56.562213 8159 raft_consensus.cc:385] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.562551 8159 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.563337 8159 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.565812 8159 ts_tablet_manager.cc:1434] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260616 19:57:56.567353 8159 tablet_bootstrap.cc:492] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:56.574008 8159 tablet_bootstrap.cc:654] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.578027 8161 raft_consensus.cc:359] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.579047 8161 raft_consensus.cc:385] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.579327 8161 raft_consensus.cc:740] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.579993 8161 consensus_queue.cc:260] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.581626 8161 ts_tablet_manager.cc:1434] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.047s user 0.005s sys 0.001s
I20260616 19:57:56.582561 8161 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:56.590392 8159 tablet_bootstrap.cc:492] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.590903 8159 ts_tablet_manager.cc:1403] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.000s
I20260616 19:57:56.591917 8161 tablet_bootstrap.cc:654] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.593729 8159 raft_consensus.cc:359] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.594631 8159 raft_consensus.cc:385] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.594903 8159 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.595606 8159 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.598985 8159 ts_tablet_manager.cc:1434] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260616 19:57:56.601292 8354 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:56.602746 8159 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:56.603819 8160 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:56.604769 8160 ts_tablet_manager.cc:1403] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.076s user 0.027s sys 0.016s
I20260616 19:57:56.606861 8322 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.607321 8322 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.608206 8160 raft_consensus.cc:359] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.609032 8160 raft_consensus.cc:385] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.609387 8160 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.609905 8322 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.611332 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.616341 8160 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.617595 7968 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:56.622671 8160 ts_tablet_manager.cc:1434] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260616 19:57:56.623561 8160 tablet_bootstrap.cc:492] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:56.619565 8011 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.630046 8225 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.620455 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.630367 8161 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:56.631496 8161 ts_tablet_manager.cc:1403] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.049s user 0.012s sys 0.009s
I20260616 19:57:56.630977 8225 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.632275 8225 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.632035 8319 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:56.633577 8118 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:56.642606 8161 raft_consensus.cc:359] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.643611 8161 raft_consensus.cc:385] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.644058 8161 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.645367 8161 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.661366 8161 ts_tablet_manager.cc:1434] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.030s user 0.008s sys 0.000s
I20260616 19:57:56.662340 8161 tablet_bootstrap.cc:492] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:56.665772 8225 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.673002 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:56.673882 7968 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.675714 8160 tablet_bootstrap.cc:654] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.688357 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.688988 8118 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.690290 8161 tablet_bootstrap.cc:654] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.693315 8322 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.693796 8322 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.695604 7968 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:56.695778 8322 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.685927 8225 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.696900 8011 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.697213 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.697822 7961 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:56.698707 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.699316 8115 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:56.699549 8159 tablet_bootstrap.cc:654] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.701368 8009 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.699681 8118 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:56.707788 8322 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:56.709499 8225 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.711606 8322 raft_consensus.cc:697] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:56.711723 8225 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.712287 8225 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.712564 8322 consensus_queue.cc:237] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.702399 8318 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.715117 8318 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.723670 8318 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.725297 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.725641 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.726148 8118 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:56.727555 7934 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:56.726423 8161 tablet_bootstrap.cc:492] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:56.728757 8161 ts_tablet_manager.cc:1403] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.067s user 0.008s sys 0.005s
I20260616 19:57:56.730294 8318 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.742758 8318 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.743242 8166 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.743301 8318 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.743862 8166 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.748721 8166 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:56.751107 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.751966 7961 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:56.752791 8225 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.753468 8085 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:56.754922 8040 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:56.764334 8166 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.765625 8166 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.765985 8166 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.765816 8225 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.772408 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.774153 8115 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.775744 8161 raft_consensus.cc:359] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.776449 8161 raft_consensus.cc:385] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.776713 8161 raft_consensus.cc:740] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.777356 8161 consensus_queue.cc:260] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.782622 8354 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.783093 8354 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.784390 8115 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:56.785571 8161 ts_tablet_manager.cc:1434] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.056s user 0.002s sys 0.003s
I20260616 19:57:56.794930 8161 tablet_bootstrap.cc:492] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:56.787371 8318 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.793852 8159 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.798568 8159 ts_tablet_manager.cc:1403] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.197s user 0.005s sys 0.023s
I20260616 19:57:56.800745 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.801395 8043 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:56.802034 8354 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.805374 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:57:56.806644 7934 leader_election.cc:343] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.810581 8159 raft_consensus.cc:359] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.812341 8159 raft_consensus.cc:385] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.812835 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.813694 8009 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:57:56.816908 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 62519us
I20260616 19:57:56.814708 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:56.817516 8116 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:56.817571 8393 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:56.817775 8043 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:56.818137 8393 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:56.818679 8159 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.818814 7935 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:57:56.819518 8159 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.818966 8393 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.812008 8318 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.815941 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:56.816838 8160 tablet_bootstrap.cc:492] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:56.819903 8354 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.822474 8159 ts_tablet_manager.cc:1434] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.024s user 0.003s sys 0.002s
I20260616 19:57:56.812790 8166 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.821368 8354 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.823321 8354 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.823730 8159 tablet_bootstrap.cc:492] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:56.825141 8166 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:56.825816 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:56.826593 7968 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:57:56.831280 8160 ts_tablet_manager.cc:1403] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.208s user 0.018s sys 0.004s
I20260616 19:57:56.855203 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:56.856285 8318 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:56.860198 8159 tablet_bootstrap.cc:654] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.860615 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:56.865895 7961 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.861235 8160 raft_consensus.cc:359] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.864517 8161 tablet_bootstrap.cc:654] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.871580 8160 raft_consensus.cc:385] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.872115 8160 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.876278 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:56.878949 8040 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.874181 7865 catalog_manager.cc:5696] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:56.885349 8393 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.886565 8161 tablet_bootstrap.cc:492] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:56.887060 8161 ts_tablet_manager.cc:1403] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.092s user 0.003s sys 0.007s
I20260616 19:57:56.887135 8160 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.886049 8393 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.889259 8160 ts_tablet_manager.cc:1434] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.058s user 0.007s sys 0.000s
I20260616 19:57:56.890157 8160 tablet_bootstrap.cc:492] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:56.890226 8161 raft_consensus.cc:359] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.890407 8354 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.890872 8161 raft_consensus.cc:385] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.892849 8161 raft_consensus.cc:740] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:56.893441 8161 consensus_queue.cc:260] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.890276 7961 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:56.886358 8040 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:56.895985 8084 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.896101 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.896595 8161 ts_tablet_manager.cc:1434] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.009s user 0.005s sys 0.002s
I20260616 19:57:56.896804 8306 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:56.897261 8306 raft_consensus.cc:697] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:57:56.897639 7934 leader_election.cc:343] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.898283 8354 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.898085 8306 consensus_queue.cc:237] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.899291 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:56.899780 8043 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.909746 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:56.910482 7961 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:56.912143 8393 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.913194 8011 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.914824 8161 tablet_bootstrap.cc:492] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:56.918354 8393 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.918293 8354 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:56.918794 8393 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.919080 8043 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:56.911039 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.921078 8116 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:56.919176 8393 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.925800 7935 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:57:56.926602 8354 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:56.927800 8354 raft_consensus.cc:697] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:56.930095 8354 consensus_queue.cc:237] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.930970 7865 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:56.932281 8399 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:56.933302 8399 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.935631 8399 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.939023 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:56.939690 8116 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:56.941022 8009 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.941804 8399 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:56.942142 8399 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:56.942456 8399 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.944456 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:56.945868 8011 leader_election.cc:343] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.948824 8161 tablet_bootstrap.cc:654] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.948980 7866 catalog_manager.cc:5696] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:56.956095 8159 tablet_bootstrap.cc:492] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:56.956709 8159 ts_tablet_manager.cc:1403] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.133s user 0.011s sys 0.005s
I20260616 19:57:56.958107 8399 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.960579 8393 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.963058 8393 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.965317 8399 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:56.965732 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.966320 8115 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.971127 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:56.971750 7968 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.976354 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:56.977177 7867 catalog_manager.cc:5696] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:56.979127 7968 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:56.979396 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:56.980028 8118 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:56.982733 8115 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:56.984969 8009 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:56.985805 8160 tablet_bootstrap.cc:654] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:56.985858 8399 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
W20260616 19:57:56.987229 8011 leader_election.cc:343] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:56.986752 8399 raft_consensus.cc:697] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:56.989915 8399 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.991928 8354 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:56.997582 8159 raft_consensus.cc:359] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:56.998474 8159 raft_consensus.cc:385] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:56.998746 8159 raft_consensus.cc:740] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.002378 8159 consensus_queue.cc:260] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.008841 8159 ts_tablet_manager.cc:1434] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.052s user 0.003s sys 0.005s
I20260616 19:57:57.010349 8159 tablet_bootstrap.cc:492] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.016513 8118 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:57.026845 8009 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:57.027776 8393 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:57.028477 8393 raft_consensus.cc:697] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:57.029246 8393 consensus_queue.cc:237] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.030848 8160 tablet_bootstrap.cc:492] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:57.031468 8160 ts_tablet_manager.cc:1403] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.142s user 0.008s sys 0.004s
I20260616 19:57:57.035457 8160 raft_consensus.cc:359] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.038378 8160 raft_consensus.cc:385] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.038755 8160 raft_consensus.cc:740] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.042353 8160 consensus_queue.cc:260] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.044575 8161 tablet_bootstrap.cc:492] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:57.045190 8161 ts_tablet_manager.cc:1403] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.131s user 0.011s sys 0.012s
I20260616 19:57:57.048593 8161 raft_consensus.cc:359] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.049008 8159 tablet_bootstrap.cc:654] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.049221 8161 raft_consensus.cc:385] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.049525 8161 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.050231 8161 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.050328 7866 catalog_manager.cc:5696] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:57.052964 8160 ts_tablet_manager.cc:1434] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.021s user 0.005s sys 0.006s
I20260616 19:57:57.054183 8160 tablet_bootstrap.cc:492] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:57.059836 8161 ts_tablet_manager.cc:1434] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.014s user 0.006s sys 0.002s
I20260616 19:57:57.060817 8161 tablet_bootstrap.cc:492] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:57.076990 8159 tablet_bootstrap.cc:492] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.077530 8159 ts_tablet_manager.cc:1403] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.068s user 0.014s sys 0.005s
I20260616 19:57:57.079535 7866 catalog_manager.cc:5696] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:57.083768 8160 tablet_bootstrap.cc:654] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.085170 8161 tablet_bootstrap.cc:654] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.087255 8159 raft_consensus.cc:359] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.088462 8159 raft_consensus.cc:385] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.088799 8159 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.089617 8159 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.093885 8159 ts_tablet_manager.cc:1434] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.016s user 0.004s sys 0.003s
I20260616 19:57:57.095204 8161 tablet_bootstrap.cc:492] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:57.095506 8159 tablet_bootstrap.cc:492] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.095631 8161 ts_tablet_manager.cc:1403] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.010s
I20260616 19:57:57.098170 8161 raft_consensus.cc:359] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.098812 8161 raft_consensus.cc:385] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.099148 8160 tablet_bootstrap.cc:492] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:57.099080 8161 raft_consensus.cc:740] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.099668 8160 ts_tablet_manager.cc:1403] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.046s user 0.014s sys 0.002s
I20260616 19:57:57.100634 8159 tablet_bootstrap.cc:654] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.099915 8161 consensus_queue.cc:260] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.102672 8160 raft_consensus.cc:359] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.103328 8160 raft_consensus.cc:385] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.103569 8160 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.104172 8160 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.106343 8160 ts_tablet_manager.cc:1434] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260616 19:57:57.107189 8160 tablet_bootstrap.cc:492] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:57.115034 8161 ts_tablet_manager.cc:1434] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.019s user 0.003s sys 0.004s
I20260616 19:57:57.117004 8160 tablet_bootstrap.cc:654] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.127321 8161 tablet_bootstrap.cc:492] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:57.128321 8160 tablet_bootstrap.cc:492] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:57.128747 8160 ts_tablet_manager.cc:1403] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.003s
I20260616 19:57:57.132454 8160 raft_consensus.cc:359] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.133091 8160 raft_consensus.cc:385] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.133347 8160 raft_consensus.cc:740] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.135705 8160 consensus_queue.cc:260] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.137439 8160 ts_tablet_manager.cc:1434] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260616 19:57:57.138293 8160 tablet_bootstrap.cc:492] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:57.146049 8160 tablet_bootstrap.cc:654] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.146176 8161 tablet_bootstrap.cc:654] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.157006 8159 tablet_bootstrap.cc:492] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.157341 8160 tablet_bootstrap.cc:492] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:57.157545 8159 ts_tablet_manager.cc:1403] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.062s user 0.020s sys 0.011s
I20260616 19:57:57.157943 8160 ts_tablet_manager.cc:1403] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.002s
W20260616 19:57:57.161087 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:57.161352 8160 raft_consensus.cc:359] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.162060 8160 raft_consensus.cc:385] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.162355 8160 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.162989 8160 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.164913 8160 ts_tablet_manager.cc:1434] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260616 19:57:57.165184 8159 raft_consensus.cc:359] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.165858 8159 raft_consensus.cc:385] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.166112 8159 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.166204 8160 tablet_bootstrap.cc:492] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:57.167198 8161 tablet_bootstrap.cc:492] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:57.166833 8159 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.168040 8161 ts_tablet_manager.cc:1403] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.008s
I20260616 19:57:57.168889 8159 ts_tablet_manager.cc:1434] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.011s user 0.005s sys 0.002s
I20260616 19:57:57.170207 8159 tablet_bootstrap.cc:492] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.171401 8161 raft_consensus.cc:359] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.172132 8161 raft_consensus.cc:385] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.172592 8161 raft_consensus.cc:740] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.173025 8160 tablet_bootstrap.cc:654] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.173699 8161 consensus_queue.cc:260] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.176716 8159 tablet_bootstrap.cc:654] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.182113 8323 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.182626 8323 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.184459 8304 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.184835 8323 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.184926 8304 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.186908 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.187481 8304 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:57.187604 8118 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:57.188988 7934 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.189663 8159 tablet_bootstrap.cc:492] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.190073 8159 ts_tablet_manager.cc:1403] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.007s
I20260616 19:57:57.191913 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.192526 8159 raft_consensus.cc:359] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.192713 8043 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:57.193172 8159 raft_consensus.cc:385] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.193487 8159 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.193476 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.193966 8393 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:57.194392 8159 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.195343 8354 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.195694 8354 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.196026 8354 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.196386 8159 ts_tablet_manager.cc:1434] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260616 19:57:57.197129 8084 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:57.197707 8159 tablet_bootstrap.cc:492] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
W20260616 19:57:57.198179 7935 leader_election.cc:343] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:57.198518 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:57.200042 8085 leader_election.cc:343] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:57.201134 8304 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.201556 8304 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.201956 8304 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.203387 8354 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.206852 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:57:57.208467 7935 leader_election.cc:343] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:57.210808 8393 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:57.211113 8354 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.216059 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:57.215842 8393 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:57.218273 8159 tablet_bootstrap.cc:654] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.216621 8118 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.237548 8161 ts_tablet_manager.cc:1434] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.069s user 0.003s sys 0.003s
I20260616 19:57:57.238909 8161 tablet_bootstrap.cc:492] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:57.241950 8118 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:57.249121 7934 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:57.250320 8354 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:57.215314 8304 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.254401 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:57.254585 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:57.255085 8043 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:57.255601 8085 leader_election.cc:343] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:57.250911 8354 raft_consensus.cc:697] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:57.264284 8304 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:57.263914 8354 consensus_queue.cc:237] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.264986 8043 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:57.270720 8084 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:57.272689 8166 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:57.273293 8166 raft_consensus.cc:697] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:57.274302 8166 consensus_queue.cc:237] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.293136 8159 tablet_bootstrap.cc:492] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.293602 8159 ts_tablet_manager.cc:1403] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.096s user 0.021s sys 0.024s
I20260616 19:57:57.301277 8399 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:57.301601 8159 raft_consensus.cc:359] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.302286 8159 raft_consensus.cc:385] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.302565 8159 raft_consensus.cc:740] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.303141 8159 consensus_queue.cc:260] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.305130 8159 ts_tablet_manager.cc:1434] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260616 19:57:57.305945 8159 tablet_bootstrap.cc:492] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.314519 8116 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.314720 7867 catalog_manager.cc:5696] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:57.318204 8159 tablet_bootstrap.cc:654] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.318145 8306 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.318742 8306 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.321236 8306 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:57.322749 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:57.323743 7961 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:57.324129 8323 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:57.325090 8085 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.326016 8306 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.326458 8306 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.326853 8306 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.331761 8159 tablet_bootstrap.cc:492] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.332340 8159 ts_tablet_manager.cc:1403] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.006s
I20260616 19:57:57.335191 8159 raft_consensus.cc:359] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.335618 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.335867 8159 raft_consensus.cc:385] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.336264 8159 raft_consensus.cc:740] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.336675 8306 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.337378 8159 consensus_queue.cc:260] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.339462 8159 ts_tablet_manager.cc:1434] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260616 19:57:57.340168 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:57.340436 8159 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.340746 7961 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:57.343209 8084 leader_election.cc:343] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:57.344826 7865 catalog_manager.cc:5696] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:57.347822 7961 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:57.348174 8160 tablet_bootstrap.cc:492] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:57.348605 8160 ts_tablet_manager.cc:1403] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.183s user 0.052s sys 0.046s
I20260616 19:57:57.348848 8159 tablet_bootstrap.cc:654] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.349399 8085 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.350842 8160 raft_consensus.cc:359] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.351388 8160 raft_consensus.cc:385] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.351655 8160 raft_consensus.cc:740] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.352442 8160 consensus_queue.cc:260] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.354885 8160 ts_tablet_manager.cc:1434] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260616 19:57:57.355516 8304 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:57.355865 8310 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:57.356443 8160 tablet_bootstrap.cc:492] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:57.357923 8161 tablet_bootstrap.cc:654] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.360038 8159 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.361585 8159 ts_tablet_manager.cc:1403] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.009s
I20260616 19:57:57.364297 8159 raft_consensus.cc:359] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.365100 8159 raft_consensus.cc:385] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.365399 8159 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.365576 8304 raft_consensus.cc:697] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:57.365736 8306 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:57.366047 8159 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.366433 8304 consensus_queue.cc:237] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.368125 8160 tablet_bootstrap.cc:654] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.369060 8159 ts_tablet_manager.cc:1434] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260616 19:57:57.370004 8159 tablet_bootstrap.cc:492] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.370806 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:57:57.372576 8084 leader_election.cc:343] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:57.376052 8159 tablet_bootstrap.cc:654] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.386047 8354 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.386947 8160 tablet_bootstrap.cc:492] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:57.386560 8354 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.387424 8160 ts_tablet_manager.cc:1403] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.005s
I20260616 19:57:57.388670 8354 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.389793 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.390167 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.390381 8118 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:57.390774 8043 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:57.390698 8160 raft_consensus.cc:359] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.391428 8160 raft_consensus.cc:385] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.391884 8160 raft_consensus.cc:740] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.391816 7934 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.393633 8160 consensus_queue.cc:260] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.396021 8160 ts_tablet_manager.cc:1434] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260616 19:57:57.395905 8354 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.396409 8354 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.396831 8354 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.407078 8161 tablet_bootstrap.cc:492] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:57.408914 8161 ts_tablet_manager.cc:1403] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.170s user 0.018s sys 0.013s
I20260616 19:57:57.411607 8161 raft_consensus.cc:359] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.412307 8161 raft_consensus.cc:385] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.412595 8161 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.412323 8354 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.416421 8225 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.416944 8225 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.417508 8160 tablet_bootstrap.cc:492] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:57.418951 8225 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.413309 8161 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.424259 8161 ts_tablet_manager.cc:1434] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260616 19:57:57.425118 8161 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:57.425153 8159 tablet_bootstrap.cc:492] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.426940 8159 ts_tablet_manager.cc:1403] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.004s
I20260616 19:57:57.425329 8319 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260616 19:57:57.432853 8159 raft_consensus.cc:359] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.433591 8323 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.433873 8354 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.434157 8323 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.435146 8159 raft_consensus.cc:385] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.435484 8159 raft_consensus.cc:740] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.437525 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.437889 8323 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.438361 8118 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:57:57.436508 8159 consensus_queue.cc:260] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.440972 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.445303 8116 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:57.445947 8159 ts_tablet_manager.cc:1434] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.019s user 0.007s sys 0.001s
I20260616 19:57:57.444877 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:57.446842 8159 tablet_bootstrap.cc:492] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.447000 8160 tablet_bootstrap.cc:654] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.446977 7968 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.449899 7866 catalog_manager.cc:5696] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:57.458894 8011 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:57.459738 8225 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.456022 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:57.460500 8118 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.460399 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.457280 8393 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.467826 8393 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.467078 8399 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.470243 8399 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.474880 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:57.496966 8040 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.500624 8310 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.501132 8310 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.510865 8354 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.510950 8400 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:57.511634 8354 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.517315 8310 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:57.521281 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:57.522207 8161 tablet_bootstrap.cc:654] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.522047 7961 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:57.525929 8354 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.527705 8085 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.496933 8431 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260616 19:57:57.528276 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.535140 8393 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.535054 7935 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:57.535629 8399 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.536439 8159 tablet_bootstrap.cc:654] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.545210 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.536295 8310 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.550411 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.551054 8310 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.551152 8043 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:57.551802 8310 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.562783 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:57.563680 7964 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:57.568106 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:57.569454 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:57.570281 8399 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.571211 8399 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.572345 8399 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.570463 7964 raft_consensus.cc:2393] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:57.599723 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.600186 8160 tablet_bootstrap.cc:492] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:57.600963 8160 ts_tablet_manager.cc:1403] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.184s user 0.009s sys 0.023s
I20260616 19:57:57.622722 8393 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260616 19:57:57.628665 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 76294.1us
I20260616 19:57:57.631207 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.631312 8393 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:57.632074 8043 raft_consensus.cc:2393] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:57.632023 8322 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.633896 8319 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.635403 8322 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.637871 8319 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.640205 8319 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.640707 8322 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.642815 8225 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:57.643883 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.644587 8117 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:57.637321 8011 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:57.646335 7934 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.647090 8323 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.647437 8323 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.648479 8160 raft_consensus.cc:359] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.649125 8160 raft_consensus.cc:385] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.649390 8160 raft_consensus.cc:740] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.650051 8160 consensus_queue.cc:260] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.647768 8323 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.655808 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.656490 8117 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:57.657577 8009 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:57.658488 8322 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.658838 8322 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.660204 8160 ts_tablet_manager.cc:1434] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.059s user 0.004s sys 0.002s
I20260616 19:57:57.646862 8225 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.668143 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:57.680733 8160 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:57.660584 8322 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.658897 7935 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:57.701519 8161 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:57.702163 8161 ts_tablet_manager.cc:1403] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.277s user 0.019s sys 0.038s
I20260616 19:57:57.705514 8319 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.706602 8161 raft_consensus.cc:359] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.707267 8161 raft_consensus.cc:385] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.707512 8161 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.708088 8161 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.718622 8161 ts_tablet_manager.cc:1434] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.016s user 0.004s sys 0.003s
I20260616 19:57:57.719542 8161 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:57.715569 8354 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:57.726864 8304 consensus_queue.cc:1048] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260616 19:57:57.748940 8399 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.753455 8040 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:57.733999 8118 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:57.771066 8161 tablet_bootstrap.cc:654] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:57.754989 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 86353.5us
I20260616 19:57:57.784085 7934 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.784726 8319 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:57.785369 8319 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:57.785822 8319 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:57.786625 8319 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.797142 8310 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.800797 8116 raft_consensus.cc:2393] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:57.801059 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:57.776551 8323 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.804474 7963 raft_consensus.cc:2393] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:57:57.805332 8323 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.810685 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:57.794992 8354 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.811565 8116 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.816015 8354 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.819754 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:57.820873 8118 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:57.822146 7934 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:57.822985 8444 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:57.823452 8444 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:57.831357 8444 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:57.832098 8159 tablet_bootstrap.cc:492] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:57.836236 8159 ts_tablet_manager.cc:1403] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.390s user 0.023s sys 0.028s
I20260616 19:57:57.812708 8322 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.845479 8160 tablet_bootstrap.cc:654] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:57.843840 8354 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.847256 8225 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:57.857656 8310 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:57.860013 8448 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.860644 8448 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.844259 8318 consensus_queue.cc:1048] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260616 19:57:57.863152 8042 raft_consensus.cc:2393] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:57.854116 8399 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:57.876102 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:57.877521 8159 raft_consensus.cc:359] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.883456 8159 raft_consensus.cc:385] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:57.883828 8159 raft_consensus.cc:740] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:57.884647 8159 consensus_queue.cc:260] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:57.886302 8159 ts_tablet_manager.cc:1434] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.050s user 0.004s sys 0.000s
I20260616 19:57:57.893827 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:57.894923 8044 raft_consensus.cc:2393] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:57.997432 8448 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.023469 8161 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:58.023998 8116 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:58.024124 8161 ts_tablet_manager.cc:1403] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.305s user 0.018s sys 0.026s
I20260616 19:57:57.950505 8114 raft_consensus.cc:2393] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:57:57.997109 8447 log.cc:927] Time spent T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.120s user 0.001s sys 0.000s
I20260616 19:57:58.032075 8159 tablet_bootstrap.cc:492] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:57.997109 8306 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:57.947906 8322 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:57:57.963420 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 65407.6us
I20260616 19:57:58.034632 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:58.048691 7964 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.052119 8444 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.052510 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:58.036957 8306 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.057590 8306 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:58.055114 8444 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.066951 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:58.069736 8040 raft_consensus.cc:2393] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:57:58.071910 7934 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:58.072759 8444 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:58.073283 8444 raft_consensus.cc:697] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:58.073709 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:58.074514 7963 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:58.074312 8444 consensus_queue.cc:237] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.086905 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:58.087643 8043 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:58.098352 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:58.099363 8455 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:57:58.099838 8455 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:58.100286 8455 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:58.100957 8319 consensus_queue.cc:1048] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.106726 8451 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.107257 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:58.106905 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:58.124780 8306 consensus_queue.cc:1048] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.125908 8160 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:58.126387 8160 ts_tablet_manager.cc:1403] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.446s user 0.009s sys 0.025s
I20260616 19:57:58.131152 8159 tablet_bootstrap.cc:654] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:58.131906 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 70631.5us
I20260616 19:57:58.137452 8455 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.137629 8160 raft_consensus.cc:359] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.138342 8160 raft_consensus.cc:385] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.138609 8160 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.139544 8160 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.132622 8116 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.149577 8160 ts_tablet_manager.cc:1434] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.023s user 0.008s sys 0.000s
I20260616 19:57:58.145553 8454 consensus_queue.cc:1048] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260616 19:57:58.150156 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:58.151597 8455 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:58.142002 8431 consensus_queue.cc:1048] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260616 19:57:58.152627 8442 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.155022 8160 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:58.153081 8442 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.149626 8161 raft_consensus.cc:359] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.157655 8161 raft_consensus.cc:385] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.157128 8442 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.158042 8161 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.158775 8161 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.161373 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:58.154182 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:58.167502 7968 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:58.170722 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:58.171368 7965 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:58.186780 8161 ts_tablet_manager.cc:1434] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.159s user 0.007s sys 0.000s
I20260616 19:57:58.187563 8161 tablet_bootstrap.cc:492] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:58.195837 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.196707 8119 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:58.197926 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:58.198237 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:58.198552 8036 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:57:58.198962 8117 raft_consensus.cc:2393] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:57:58.219311 7935 leader_election.cc:343] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.189206 7964 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:58.241462 7865 catalog_manager.cc:5696] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:58.243913 7865 catalog_manager.cc:5696] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:58.188365 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:58.249661 8304 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.257045 8400 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260616 19:57:58.264815 7935 leader_election.cc:343] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.266273 8085 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:58.269542 8161 tablet_bootstrap.cc:654] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:58.278678 8460 log.cc:927] Time spent T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.094s user 0.001s sys 0.000s
I20260616 19:57:58.254690 8304 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.281963 8304 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:58.282392 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.283897 8455 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
W20260616 19:57:58.270514 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 86061us
I20260616 19:57:58.284909 8455 raft_consensus.cc:697] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:58.300420 8011 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:58.310628 8160 tablet_bootstrap.cc:654] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
W20260616 19:57:58.311691 8009 leader_election.cc:343] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.312425 8455 consensus_queue.cc:237] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.325860 8116 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:58.328251 7965 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:57:58.329674 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:58.330367 8038 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:58.332911 8085 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:58.320442 8451 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:57:58.341372 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:58.343050 8458 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:57:58.343566 8458 raft_consensus.cc:697] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:58.346094 8084 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:58.346983 8319 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.340718 7934 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:58.337566 8011 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:57:58.349964 8306 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:58.362134 8306 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:58.362481 8306 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.364953 8036 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:57:58.365701 8393 raft_consensus.cc:2764] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260616 19:57:58.368093 8085 leader_election.cc:343] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.368594 8161 tablet_bootstrap.cc:492] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:58.369024 8161 ts_tablet_manager.cc:1403] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.182s user 0.011s sys 0.038s
I20260616 19:57:58.344457 8458 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.377952 8318 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:58.368453 8319 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:58.390151 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 53175.5us
I20260616 19:57:58.394146 8318 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:58.414618 8160 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:58.415159 8160 ts_tablet_manager.cc:1403] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.261s user 0.030s sys 0.012s
I20260616 19:57:58.421260 8161 raft_consensus.cc:359] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.422297 8161 raft_consensus.cc:385] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.422647 8319 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.422668 8161 raft_consensus.cc:740] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.423839 8161 consensus_queue.cc:260] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.427096 8159 tablet_bootstrap.cc:492] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:58.427935 8159 ts_tablet_manager.cc:1403] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.396s user 0.024s sys 0.011s
I20260616 19:57:58.432520 8161 ts_tablet_manager.cc:1434] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.063s user 0.008s sys 0.001s
I20260616 19:57:58.436625 8318 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.437441 8159 raft_consensus.cc:359] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.438467 8159 raft_consensus.cc:385] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.438855 8159 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.439713 8159 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.442698 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:58.445848 8041 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:58.446986 8160 raft_consensus.cc:359] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.448468 8160 raft_consensus.cc:385] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.448861 8160 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.454422 8306 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:58.469522 8475 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.052s user 0.003s sys 0.002s
I20260616 19:57:58.473891 8160 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.484192 8159 ts_tablet_manager.cc:1434] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.054s user 0.007s sys 0.001s
I20260616 19:57:58.485183 8159 tablet_bootstrap.cc:492] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:58.485476 8160 ts_tablet_manager.cc:1434] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.070s user 0.006s sys 0.003s
I20260616 19:57:58.463079 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.508940 8161 tablet_bootstrap.cc:492] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:58.510016 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:58.510226 8393 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.494679 8113 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:58.510694 8393 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.513942 8393 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.514083 7934 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:58.515409 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:58.516579 8354 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:58.517230 8444 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:58.517632 8444 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:58.517948 8444 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:57:58.518988 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260616 19:57:58.523432 8085 leader_election.cc:343] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.535143 8306 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:58.536959 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:58.531131 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.540589 8118 raft_consensus.cc:2393] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:57:58.548516 8009 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:58.549705 8472 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260616 19:57:58.549978 8442 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.560348 8160 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:58.581846 8444 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.582996 8160 tablet_bootstrap.cc:654] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.587049 8159 tablet_bootstrap.cc:654] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.587661 7865 catalog_manager.cc:5696] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:58.589165 7865 catalog_manager.cc:5696] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 2, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:58.588269 8442 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:58.594475 7967 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:57:58.594740 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:58.595377 8036 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.601271 8161 tablet_bootstrap.cc:654] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.602360 8011 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:58.603506 8400 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.608907 8323 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.609632 8323 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.622066 8323 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.623190 8160 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:58.623654 8160 ts_tablet_manager.cc:1403] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.064s user 0.010s sys 0.013s
I20260616 19:57:58.626904 8160 raft_consensus.cc:359] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.628757 8160 raft_consensus.cc:385] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.627068 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.629158 8160 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.629860 8160 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.632205 8160 ts_tablet_manager.cc:1434] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.008s user 0.006s sys 0.003s
I20260616 19:57:58.633543 8160 tablet_bootstrap.cc:492] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
W20260616 19:57:58.634696 7934 leader_election.cc:343] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.635727 8400 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:58.620059 7866 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:58.636142 8354 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:58.637526 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:58.638859 7935 leader_election.cc:343] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.639226 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:58.640280 8319 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:58.648082 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:58.646522 8036 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:58.653270 8084 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:58.655143 8455 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:58.655660 8455 raft_consensus.cc:697] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:58.656571 8455 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.659529 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:58.660590 8117 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.667711 8444 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.669726 8159 tablet_bootstrap.cc:492] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:58.670153 8159 ts_tablet_manager.cc:1403] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.185s user 0.011s sys 0.017s
I20260616 19:57:58.680917 8117 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:58.688637 8159 raft_consensus.cc:359] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.689474 8159 raft_consensus.cc:385] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.689798 8159 raft_consensus.cc:740] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.690505 8159 consensus_queue.cc:260] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.696869 7934 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:58.697819 8323 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:58.698370 8323 raft_consensus.cc:697] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:58.699275 8323 consensus_queue.cc:237] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.700222 8318 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.703476 8159 ts_tablet_manager.cc:1434] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.033s user 0.006s sys 0.001s
I20260616 19:57:58.706765 8160 tablet_bootstrap.cc:654] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.716439 8159 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:58.722384 8037 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:58.735895 7865 catalog_manager.cc:5696] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:57:58.750240 8161 tablet_bootstrap.cc:492] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:58.750768 8161 ts_tablet_manager.cc:1403] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.242s user 0.018s sys 0.013s
I20260616 19:57:58.754402 7867 catalog_manager.cc:5696] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:58.756392 8161 raft_consensus.cc:359] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.757067 8161 raft_consensus.cc:385] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.757360 8161 raft_consensus.cc:740] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.757952 8161 consensus_queue.cc:260] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.759604 8159 tablet_bootstrap.cc:654] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.761128 8306 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:58.766345 8161 ts_tablet_manager.cc:1434] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260616 19:57:58.772770 8161 tablet_bootstrap.cc:492] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:58.781526 8310 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.785100 8161 tablet_bootstrap.cc:654] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.791566 8159 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:58.792354 8159 ts_tablet_manager.cc:1403] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.076s user 0.008s sys 0.007s
I20260616 19:57:58.794652 8159 raft_consensus.cc:359] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.795190 8159 raft_consensus.cc:385] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.795444 8159 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.796052 8159 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.798827 8159 ts_tablet_manager.cc:1434] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260616 19:57:58.799779 8159 tablet_bootstrap.cc:492] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:58.803316 7961 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260616 19:57:58.803419 8161 tablet_bootstrap.cc:492] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:58.806301 8161 ts_tablet_manager.cc:1403] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.004s
I20260616 19:57:58.806548 8159 tablet_bootstrap.cc:654] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.811162 8455 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.818562 8159 tablet_bootstrap.cc:492] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:58.818790 8354 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.819026 8159 ts_tablet_manager.cc:1403] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.000s
I20260616 19:57:58.811630 8161 raft_consensus.cc:359] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.819533 8161 raft_consensus.cc:385] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.819227 8354 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.819857 8161 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.820717 8161 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.822321 8161 ts_tablet_manager.cc:1434] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260616 19:57:58.822018 8159 raft_consensus.cc:359] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.822220 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:58.822715 8159 raft_consensus.cc:385] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.823010 8159 raft_consensus.cc:740] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.823036 8040 raft_consensus.cc:2393] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:57:58.823378 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.824961 7934 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:58.824846 8159 consensus_queue.cc:260] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.825750 8444 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.826792 8159 ts_tablet_manager.cc:1434] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260616 19:57:58.827665 8159 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:58.823195 8161 tablet_bootstrap.cc:492] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:58.834540 8354 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.834782 8040 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260616 19:57:58.836342 8444 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:58.836575 8306 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.840708 8159 tablet_bootstrap.cc:654] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.843145 8161 tablet_bootstrap.cc:654] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.852577 8444 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.854537 8444 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.868324 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:58.872108 7935 leader_election.cc:343] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.873879 8354 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:58.866927 8160 tablet_bootstrap.cc:492] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:58.881666 8160 ts_tablet_manager.cc:1403] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.248s user 0.012s sys 0.039s
I20260616 19:57:58.882800 8159 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:58.886301 8159 ts_tablet_manager.cc:1403] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.059s user 0.011s sys 0.002s
I20260616 19:57:58.894958 8160 raft_consensus.cc:359] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.895515 8444 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.895787 8160 raft_consensus.cc:385] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.896095 8160 raft_consensus.cc:740] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.896962 8161 tablet_bootstrap.cc:492] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:58.897389 8161 ts_tablet_manager.cc:1403] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.074s user 0.011s sys 0.006s
I20260616 19:57:58.897564 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.897810 8160 consensus_queue.cc:260] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:58.900488 7934 leader_election.cc:343] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.900571 8159 raft_consensus.cc:359] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.901471 8159 raft_consensus.cc:385] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.901147 7934 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:58.901743 8159 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.902544 8354 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:58.902499 8159 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.905138 8160 ts_tablet_manager.cc:1434] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.023s user 0.005s sys 0.002s
I20260616 19:57:58.905910 8160 tablet_bootstrap.cc:492] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:58.909950 8159 ts_tablet_manager.cc:1434] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260616 19:57:58.909996 8161 raft_consensus.cc:359] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.910952 8161 raft_consensus.cc:385] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.911311 8161 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.911409 8159 tablet_bootstrap.cc:492] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:58.922168 8161 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.928764 8161 ts_tablet_manager.cc:1434] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.031s user 0.006s sys 0.002s
I20260616 19:57:58.944126 8323 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:58.945251 8161 tablet_bootstrap.cc:492] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:58.944808 8318 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260616 19:57:58.947675 8323 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.950153 8323 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:58.951375 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:58.957201 8159 tablet_bootstrap.cc:654] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.958611 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:57:58.960397 7935 leader_election.cc:343] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.961404 8117 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:58.965696 7934 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:58.966405 8444 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:58.966845 8444 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:58.967239 8444 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.967994 8160 tablet_bootstrap.cc:654] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.969343 8161 tablet_bootstrap.cc:654] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:58.975579 8161 tablet_bootstrap.cc:492] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:58.976020 8161 ts_tablet_manager.cc:1403] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.039s user 0.008s sys 0.001s
I20260616 19:57:58.978554 8161 raft_consensus.cc:359] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.979233 8161 raft_consensus.cc:385] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.979523 8161 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.976338 8444 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.984874 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:58.985373 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:58.985827 8160 tablet_bootstrap.cc:492] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:58.985951 8117 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:58.986268 8160 ts_tablet_manager.cc:1403] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.080s user 0.011s sys 0.004s
W20260616 19:57:58.991786 7935 leader_election.cc:343] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:58.993678 8160 raft_consensus.cc:359] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.994361 8160 raft_consensus.cc:385] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:58.994611 8160 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:58.995860 8117 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:58.996855 8161 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.998476 7934 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:58.999521 8323 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.000002 8323 raft_consensus.cc:697] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:59.000622 8444 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.000814 8323 consensus_queue.cc:237] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:58.995705 8160 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.004709 8160 ts_tablet_manager.cc:1434] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20260616 19:57:59.005600 8160 tablet_bootstrap.cc:492] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:59.011657 8160 tablet_bootstrap.cc:654] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.014063 8159 tablet_bootstrap.cc:492] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:59.014586 8159 ts_tablet_manager.cc:1403] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.104s user 0.024s sys 0.002s
I20260616 19:57:59.015013 8161 ts_tablet_manager.cc:1434] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260616 19:57:59.016795 8161 tablet_bootstrap.cc:492] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.023247 8161 tablet_bootstrap.cc:654] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.028486 8160 tablet_bootstrap.cc:492] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.029086 8160 ts_tablet_manager.cc:1403] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.000s
I20260616 19:57:59.031566 8306 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.028715 8159 raft_consensus.cc:359] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.032058 8306 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.032627 8159 raft_consensus.cc:385] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.033020 8159 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.033708 8159 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.034301 8306 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:59.035038 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.034145 8160 raft_consensus.cc:359] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.035790 8160 raft_consensus.cc:385] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.036176 8160 raft_consensus.cc:740] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.036825 8159 ts_tablet_manager.cc:1434] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
W20260616 19:57:59.038074 8085 leader_election.cc:343] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.036942 8160 consensus_queue.cc:260] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.038610 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:59.039314 8161 tablet_bootstrap.cc:492] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:59.039799 8159 tablet_bootstrap.cc:492] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:59.039824 8161 ts_tablet_manager.cc:1403] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.012s
W20260616 19:57:59.040264 8084 leader_election.cc:343] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.040663 8084 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:59.041296 8306 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:59.046610 7865 catalog_manager.cc:5696] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:59.041319 8160 ts_tablet_manager.cc:1434] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260616 19:57:59.048612 8160 tablet_bootstrap.cc:492] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:59.049683 8161 raft_consensus.cc:359] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.050354 8161 raft_consensus.cc:385] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.050614 8161 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.051241 8161 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.053403 8457 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:59.058166 8160 tablet_bootstrap.cc:654] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.059733 8159 tablet_bootstrap.cc:654] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.062078 8161 ts_tablet_manager.cc:1434] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260616 19:57:59.062994 8161 tablet_bootstrap.cc:492] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.068830 8161 tablet_bootstrap.cc:654] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.069582 8160 tablet_bootstrap.cc:492] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.070021 8160 ts_tablet_manager.cc:1403] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.002s
I20260616 19:57:59.072690 8160 raft_consensus.cc:359] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.073350 8160 raft_consensus.cc:385] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.073657 8160 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.075209 8160 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.077869 8159 tablet_bootstrap.cc:492] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:59.078117 8160 ts_tablet_manager.cc:1434] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260616 19:57:59.078318 8159 ts_tablet_manager.cc:1403] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.011s
I20260616 19:57:59.078958 8160 tablet_bootstrap.cc:492] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:59.080905 8159 raft_consensus.cc:359] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.081411 8451 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.081583 8159 raft_consensus.cc:385] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.081907 8159 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.081897 8451 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.082795 8159 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.084780 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.085366 7967 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.085573 8159 ts_tablet_manager.cc:1434] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260616 19:57:59.085378 8451 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.086470 8159 tablet_bootstrap.cc:492] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:59.086519 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.087248 8118 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.086423 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.092738 8160 tablet_bootstrap.cc:654] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.093210 8159 tablet_bootstrap.cc:654] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.094264 8393 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.094657 8393 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.094973 8393 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.100171 8451 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.100647 8451 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.102624 8451 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.103032 8451 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.103412 8451 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.105499 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.106163 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.106276 8118 raft_consensus.cc:2393] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:57:59.111194 8159 tablet_bootstrap.cc:492] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:59.111661 8159 ts_tablet_manager.cc:1403] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.005s
I20260616 19:57:59.121272 8011 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:59.124084 8159 raft_consensus.cc:359] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.124837 8159 raft_consensus.cc:385] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.125150 8159 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.128813 8159 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.123961 8442 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.136735 8160 tablet_bootstrap.cc:492] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.127902 8393 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.137210 8160 ts_tablet_manager.cc:1403] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.009s
I20260616 19:57:59.137677 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.138851 8393 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.140134 8160 raft_consensus.cc:359] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.140677 8159 ts_tablet_manager.cc:1434] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.029s user 0.007s sys 0.001s
I20260616 19:57:59.140957 8160 raft_consensus.cc:385] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.141265 8160 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.141944 8159 tablet_bootstrap.cc:492] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:59.142601 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.143219 7967 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.143885 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:59.144898 7961 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.146291 8160 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.148324 8160 ts_tablet_manager.cc:1434] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
I20260616 19:57:59.149150 8160 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
W20260616 19:57:59.157205 8009 leader_election.cc:343] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.158896 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:59.159489 8118 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.135915 8451 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.163368 8011 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:57:59.166247 7961 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.156103 8442 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:59.167567 8442 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.167920 8442 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.168291 8442 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.168603 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.169116 8400 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.168313 8455 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:59.169570 8400 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.171561 8400 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.175155 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.175295 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.172039 8400 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.175980 8117 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.176479 8400 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:59.178725 8400 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.177299 8009 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.180469 8318 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:57:59.180727 8451 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.181136 8451 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.181452 8451 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.182636 8118 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.175980 7965 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.190469 8161 tablet_bootstrap.cc:492] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:59.191388 8161 ts_tablet_manager.cc:1403] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.129s user 0.021s sys 0.022s
I20260616 19:57:59.194505 8159 tablet_bootstrap.cc:654] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.201597 8161 raft_consensus.cc:359] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.204291 8161 raft_consensus.cc:385] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.204648 8161 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.207538 8161 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.216516 8161 ts_tablet_manager.cc:1434] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
I20260616 19:57:59.218577 8161 tablet_bootstrap.cc:492] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.223021 8160 tablet_bootstrap.cc:654] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.224043 8451 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.230808 8318 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:57:59.225047 8442 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.235229 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:59.235858 8113 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.237169 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:59.237917 7965 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.235816 8451 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.239920 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:59.240526 7967 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.240888 8442 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.244501 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:57:59.245833 8009 leader_election.cc:343] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.247401 7965 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.253180 8160 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.254158 8160 ts_tablet_manager.cc:1403] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.105s user 0.010s sys 0.003s
I20260616 19:57:59.254289 8011 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.255781 8400 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.256254 8400 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:59.257066 8400 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.248065 8161 tablet_bootstrap.cc:654] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.216907 7867 catalog_manager.cc:5696] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:59.262619 8160 raft_consensus.cc:359] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.266486 8160 raft_consensus.cc:385] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.250278 8113 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.270017 8455 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.270704 8160 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.249711 7967 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.270504 8455 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.272315 8011 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:57:59.272181 8160 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.273452 8442 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.273989 8455 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:59.274466 8442 raft_consensus.cc:697] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:59.275519 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:59.275278 8442 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.276413 8160 ts_tablet_manager.cc:1434] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.022s user 0.008s sys 0.001s
W20260616 19:57:59.277302 8084 leader_election.cc:343] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.278709 8160 tablet_bootstrap.cc:492] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:59.278952 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:59.292438 8085 leader_election.cc:343] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.292984 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:59.294065 8306 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:59.307646 8161 tablet_bootstrap.cc:492] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:59.308193 8161 ts_tablet_manager.cc:1403] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.090s user 0.016s sys 0.008s
I20260616 19:57:59.309759 8160 tablet_bootstrap.cc:654] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.311090 8161 raft_consensus.cc:359] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.311782 8161 raft_consensus.cc:385] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.312057 8161 raft_consensus.cc:740] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.312701 8161 consensus_queue.cc:260] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.314754 8161 ts_tablet_manager.cc:1434] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260616 19:57:59.319401 8161 tablet_bootstrap.cc:492] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.325356 8161 tablet_bootstrap.cc:654] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.323817 7868 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:59.326471 8160 tablet_bootstrap.cc:492] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.327529 8160 ts_tablet_manager.cc:1403] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.049s user 0.009s sys 0.010s
I20260616 19:57:59.326877 8159 tablet_bootstrap.cc:492] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:59.328112 8159 ts_tablet_manager.cc:1403] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.186s user 0.030s sys 0.014s
I20260616 19:57:59.330622 8160 raft_consensus.cc:359] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.328434 7868 catalog_manager.cc:5696] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:59.331358 8160 raft_consensus.cc:385] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.331669 8160 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.332446 8160 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.345891 8161 tablet_bootstrap.cc:492] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:59.346429 8161 ts_tablet_manager.cc:1403] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.005s
I20260616 19:57:59.349200 8161 raft_consensus.cc:359] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.351717 8161 raft_consensus.cc:385] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.352025 8161 raft_consensus.cc:740] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.352669 8161 consensus_queue.cc:260] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.354719 8161 ts_tablet_manager.cc:1434] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260616 19:57:59.354908 8160 ts_tablet_manager.cc:1434] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.027s user 0.005s sys 0.009s
I20260616 19:57:59.355732 8160 tablet_bootstrap.cc:492] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:59.355984 8161 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.361406 8160 tablet_bootstrap.cc:654] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.359328 8159 raft_consensus.cc:359] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.362054 8161 tablet_bootstrap.cc:654] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.362342 8159 raft_consensus.cc:385] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.362680 8159 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.363422 8159 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.370890 8159 ts_tablet_manager.cc:1434] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260616 19:57:59.371757 8159 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:59.378166 8161 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:59.378789 8161 ts_tablet_manager.cc:1403] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.002s
I20260616 19:57:59.382896 8161 raft_consensus.cc:359] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.383806 8161 raft_consensus.cc:385] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.384173 8161 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.385016 8161 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.396032 8161 ts_tablet_manager.cc:1434] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.017s user 0.008s sys 0.003s
I20260616 19:57:59.396279 8442 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.396991 8161 tablet_bootstrap.cc:492] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.396790 8442 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.398831 8442 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.399724 8354 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.400391 8354 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.401723 8160 tablet_bootstrap.cc:492] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.399681 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.402103 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.402242 8160 ts_tablet_manager.cc:1403] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.024s
I20260616 19:57:59.402679 8118 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.403267 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:59.403867 8037 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:59.404038 8009 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.402271 7961 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.404910 8160 raft_consensus.cc:359] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.405604 8160 raft_consensus.cc:385] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.405874 8160 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.406014 8161 tablet_bootstrap.cc:654] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.405915 7935 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.406688 8160 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.407341 8444 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.407719 8444 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.408020 8444 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.408524 8160 ts_tablet_manager.cc:1434] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260616 19:57:59.409763 8160 tablet_bootstrap.cc:492] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:59.407347 8451 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.412851 8451 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.413219 8451 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.412873 8457 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.417412 8160 tablet_bootstrap.cc:654] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.417342 8457 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.419436 8457 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:59.420835 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:59.421444 8039 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:59.421816 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.422569 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.422623 7961 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:57:59.424857 8306 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.425192 8306 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.425510 8306 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.426801 8161 tablet_bootstrap.cc:492] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:59.427312 8161 ts_tablet_manager.cc:1403] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.006s
I20260616 19:57:59.429723 8160 tablet_bootstrap.cc:492] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.430136 8160 ts_tablet_manager.cc:1403] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.003s
I20260616 19:57:59.430413 8161 raft_consensus.cc:359] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.431104 8161 raft_consensus.cc:385] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.431417 8161 raft_consensus.cc:740] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.432122 8161 consensus_queue.cc:260] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.433310 8160 raft_consensus.cc:359] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.434072 8160 raft_consensus.cc:385] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.434228 8161 ts_tablet_manager.cc:1434] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260616 19:57:59.434324 8160 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.435395 8161 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.434967 8160 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.440593 8160 ts_tablet_manager.cc:1434] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260616 19:57:59.441931 8160 tablet_bootstrap.cc:492] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:57:59.444571 8159 tablet_bootstrap.cc:654] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.448513 8354 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.449538 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.450189 8306 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:59.450964 7934 leader_election.cc:343] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.452189 8451 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.454114 8451 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.455041 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:59.455580 8113 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.456696 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:59.457283 7961 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.458104 8306 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:59.459774 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:59.462899 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:59.463496 7967 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.461254 8039 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.477322 8113 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.480116 8009 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.481972 8451 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.482573 8451 raft_consensus.cc:697] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:57:59.483589 8451 consensus_queue.cc:237] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.493127 8442 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.518457 8161 tablet_bootstrap.cc:654] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.499743 7967 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:59.474174 8444 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.526427 8160 tablet_bootstrap.cc:654] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:57:59.526894 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:59.523157 8039 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:57:59.517400 7961 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.499836 8442 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:59.529323 7934 leader_election.cc:343] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.529400 8085 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:57:59.528676 8444 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.532477 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:59.533234 8040 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.532122 8306 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.539960 8455 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.538242 8306 raft_consensus.cc:697] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:57:59.541906 8455 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.542430 8306 consensus_queue.cc:237] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.545660 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:57:59.546904 8085 leader_election.cc:343] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.544140 8455 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:59.550007 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:57:59.550891 8040 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:59.551522 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.550464 8442 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:57:59.552104 8011 leader_election.cc:343] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.552201 8113 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:57:59.553563 8009 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:57:59.554121 7935 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:57:59.554802 8354 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.554514 8442 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.555214 8354 raft_consensus.cc:697] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:57:59.555409 8442 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.555830 8442 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.569768 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:59.565780 8354 consensus_queue.cc:237] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.574087 7866 catalog_manager.cc:5696] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:59.595801 8159 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:57:59.596472 8159 ts_tablet_manager.cc:1403] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.225s user 0.005s sys 0.023s
I20260616 19:57:59.581653 8444 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.590458 8442 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.602311 8161 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:57:59.602913 8161 ts_tablet_manager.cc:1403] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.168s user 0.007s sys 0.007s
W20260616 19:57:59.604218 8084 leader_election.cc:343] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.604739 8084 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:57:59.601490 8444 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.611204 8444 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.612834 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:59.613447 8041 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:59.614224 8159 raft_consensus.cc:359] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.614748 7935 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.614920 8159 raft_consensus.cc:385] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.615511 8318 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:57:59.615811 8159 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.615990 8318 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:57:59.616475 8442 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.616516 8318 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.617592 8159 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.618408 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.607055 8455 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:57:59.619666 8159 ts_tablet_manager.cc:1434] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260616 19:57:59.620009 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:57:59.621497 8159 tablet_bootstrap.cc:492] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:57:59.621479 8118 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.619010 8113 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:57:59.642620 8118 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:57:59.643666 8009 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:57:59.645617 8451 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:57:59.635627 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:57:59.643925 8537 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:57:59.674185 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 53337.8us
I20260616 19:57:59.626710 8541 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
I20260616 19:57:59.646173 8451 raft_consensus.cc:697] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:57:59.687369 8011 leader_election.cc:343] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:57:59.681488 8323 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.688902 8306 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.677616 8537 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.710929 8451 consensus_queue.cc:237] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:57:59.718075 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:57:59.721942 8537 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:59.716277 8306 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.723675 7867 catalog_manager.cc:5696] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:57:59.721351 8442 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:57:59.735302 8442 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.695395 8323 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.752197 8318 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.752590 8306 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:57:59.767583 8323 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:57:59.866043 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 119443us
I20260616 19:57:59.880405 8160 tablet_bootstrap.cc:492] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:57:59.881006 8160 ts_tablet_manager.cc:1403] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.440s user 0.009s sys 0.013s
W20260616 19:57:59.894327 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 139387us
I20260616 19:57:59.893546 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:57:59.904022 8161 raft_consensus.cc:359] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.871788 8354 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260616 19:57:59.914427 8114 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.905951 8161 raft_consensus.cc:385] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:57:59.923084 8161 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:57:59.923866 8161 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:57:59.925688 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:57:59.926136 8043 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:57:59.905238 8442 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:57:59.939558 8043 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:57:59.946936 8161 ts_tablet_manager.cc:1434] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.344s user 0.029s sys 0.019s
I20260616 19:57:59.948705 8161 tablet_bootstrap.cc:492] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:57:59.950559 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:57:59.951388 8118 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:58:00.023118 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:00.024448 8039 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:58:00.026657 8549 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:00.029693 7935 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:00.036855 8159 tablet_bootstrap.cc:654] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.040345 8323 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:00.043082 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:00.043510 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:00.043987 8113 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:00.041000 8323 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:00.059270 8113 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:00.059321 8442 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:00.061123 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:00.062618 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:00.063262 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:00.063812 8113 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:00.064411 7963 raft_consensus.cc:2393] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:00.075143 8444 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:00.075611 8444 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:00.075996 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:00.077607 8323 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.084520 8160 raft_consensus.cc:359] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.075996 8444 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:58:00.087036 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 43890.6us
I20260616 19:58:00.089090 8161 tablet_bootstrap.cc:654] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.087865 7866 catalog_manager.cc:5696] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:00.086374 8160 raft_consensus.cc:385] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.090250 8160 raft_consensus.cc:740] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
W20260616 19:58:00.098338 8555 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.135s user 0.005s sys 0.000s
I20260616 19:58:00.093462 8160 consensus_queue.cc:260] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.107961 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:00.108402 8009 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:00.108920 8044 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:00.116370 8442 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:00.126291 8442 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:00.126807 8442 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:00.126610 8551 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:00.140295 8160 ts_tablet_manager.cc:1434] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.259s user 0.012s sys 0.000s
I20260616 19:58:00.142724 8159 tablet_bootstrap.cc:492] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:00.143815 8159 ts_tablet_manager.cc:1403] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.522s user 0.036s sys 0.020s
I20260616 19:58:00.113679 8557 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260616 19:58:00.149542 8318 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:00.153393 8085 leader_election.cc:343] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:00.188526 8550 consensus_queue.cc:1048] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260616 19:58:00.189081 8160 tablet_bootstrap.cc:492] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:00.192312 8084 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:00.193763 8161 tablet_bootstrap.cc:492] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:00.194379 8161 ts_tablet_manager.cc:1403] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.246s user 0.011s sys 0.009s
I20260616 19:58:00.194588 8084 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:00.200100 8543 raft_consensus.cc:2804] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:00.200650 8543 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:00.201107 8543 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:00.191236 8444 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.244247 8306 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:00.250607 7961 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:58:00.347508 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 94386.5us
W20260616 19:58:00.351590 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49783.6us
I20260616 19:58:00.351280 8161 raft_consensus.cc:359] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.353652 8161 raft_consensus.cc:385] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.353950 8161 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.354790 8161 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:00.351666 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 154797us
I20260616 19:58:00.377039 8442 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.380055 8442 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:00.380914 8550 consensus_queue.cc:1048] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:00.381245 8161 ts_tablet_manager.cc:1434] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.187s user 0.006s sys 0.042s
I20260616 19:58:00.382865 8161 tablet_bootstrap.cc:492] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:00.382505 8543 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.385128 8543 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:00.385540 8451 consensus_queue.cc:1048] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:00.386123 8444 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:00.389640 8553 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:00.397682 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:00.398662 8117 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:58:00.420343 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 60949.9us
W20260616 19:58:00.421041 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 50979.5us
I20260616 19:58:00.422032 8556 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:00.437734 7867 catalog_manager.cc:5696] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:00.446134 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:00.478662 7966 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:00.480989 8160 tablet_bootstrap.cc:654] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.486086 8550 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:00.462716 8117 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:00.495252 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:00.496404 8042 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:00.498492 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:00.499347 8038 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:58:00.510149 8011 consensus_peers.cc:597] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 -> Peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Couldn't send request to peer b022cac8cb5e46b7a39a748e81e257a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:00.513978 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:00.514842 8114 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:00.487867 8159 raft_consensus.cc:359] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.519052 8159 raft_consensus.cc:385] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.519418 8159 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.522787 7867 catalog_manager.cc:5696] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:00.525733 8114 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:00.527691 8550 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:00.541384 8161 tablet_bootstrap.cc:654] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.543565 8009 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:00.544901 8393 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:00.545447 8393 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:00.546357 8393 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.554831 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:00.531433 8159 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.558693 8159 ts_tablet_manager.cc:1434] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.414s user 0.009s sys 0.014s
I20260616 19:58:00.559860 8159 tablet_bootstrap.cc:492] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:00.566437 8160 tablet_bootstrap.cc:492] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:00.566962 8160 ts_tablet_manager.cc:1403] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.378s user 0.016s sys 0.010s
I20260616 19:58:00.569653 8160 raft_consensus.cc:359] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.584318 8160 raft_consensus.cc:385] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.584419 8042 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:00.584656 8160 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.585405 8160 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.580343 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:00.587896 8542 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:58:00.591537 8542 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:00.595539 8542 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:00.595899 8544 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:00.600395 8161 tablet_bootstrap.cc:492] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:00.600768 8160 ts_tablet_manager.cc:1434] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260616 19:58:00.601857 8159 tablet_bootstrap.cc:654] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.610306 8161 ts_tablet_manager.cc:1403] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.228s user 0.013s sys 0.002s
I20260616 19:58:00.605028 8084 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:00.613579 8537 raft_consensus.cc:2804] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:00.614195 8537 raft_consensus.cc:697] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:00.610878 8160 tablet_bootstrap.cc:492] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:00.615056 8537 consensus_queue.cc:237] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.622723 8160 tablet_bootstrap.cc:654] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.623405 8161 raft_consensus.cc:359] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.624357 8161 raft_consensus.cc:385] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.624658 8161 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
W20260616 19:58:00.613606 8085 leader_election.cc:343] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:00.630541 8161 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.634573 8160 tablet_bootstrap.cc:492] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:00.635098 8160 ts_tablet_manager.cc:1403] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.009s
I20260616 19:58:00.637703 8160 raft_consensus.cc:359] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.638372 8160 raft_consensus.cc:385] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.638628 8160 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.639050 8161 ts_tablet_manager.cc:1434] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260616 19:58:00.639267 8160 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.640184 8161 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:00.641988 8160 ts_tablet_manager.cc:1434] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260616 19:58:00.642951 8160 tablet_bootstrap.cc:492] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:00.647868 8160 tablet_bootstrap.cc:654] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.650760 8318 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:00.655799 7866 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:00.657416 8159 tablet_bootstrap.cc:492] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:00.657918 8159 ts_tablet_manager.cc:1403] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.098s user 0.011s sys 0.003s
I20260616 19:58:00.658672 8160 tablet_bootstrap.cc:492] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:00.659111 8160 ts_tablet_manager.cc:1403] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260616 19:58:00.661698 8160 raft_consensus.cc:359] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.662333 8160 raft_consensus.cc:385] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.662590 8160 raft_consensus.cc:740] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.663375 8160 consensus_queue.cc:260] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.665778 8160 ts_tablet_manager.cc:1434] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260616 19:58:00.666854 8160 tablet_bootstrap.cc:492] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:00.668363 8161 tablet_bootstrap.cc:654] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.669046 8159 raft_consensus.cc:359] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.672055 8159 raft_consensus.cc:385] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.672480 8159 raft_consensus.cc:740] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.674185 8160 tablet_bootstrap.cc:654] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.673939 8159 consensus_queue.cc:260] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.679553 8159 ts_tablet_manager.cc:1434] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260616 19:58:00.680503 8159 tablet_bootstrap.cc:492] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:00.688215 8161 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:00.688751 8161 ts_tablet_manager.cc:1403] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.049s user 0.013s sys 0.005s
I20260616 19:58:00.690961 8159 tablet_bootstrap.cc:654] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.697403 8160 tablet_bootstrap.cc:492] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:00.696789 8161 raft_consensus.cc:359] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.697875 8160 ts_tablet_manager.cc:1403] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.003s
I20260616 19:58:00.697948 8161 raft_consensus.cc:385] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.698514 8161 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.699651 8161 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.700793 8160 raft_consensus.cc:359] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.701486 8160 raft_consensus.cc:385] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.701752 8160 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.702481 8160 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.703850 8161 ts_tablet_manager.cc:1434] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.015s user 0.009s sys 0.000s
I20260616 19:58:00.704583 8160 ts_tablet_manager.cc:1434] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.005s sys 0.003s
I20260616 19:58:00.705533 8161 tablet_bootstrap.cc:492] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:00.705588 8160 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:00.712332 8160 tablet_bootstrap.cc:654] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.721494 8161 tablet_bootstrap.cc:654] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.734800 8161 tablet_bootstrap.cc:492] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:00.735467 8161 ts_tablet_manager.cc:1403] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.030s user 0.012s sys 0.009s
I20260616 19:58:00.739097 8161 raft_consensus.cc:359] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.740255 8161 raft_consensus.cc:385] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:00.740685 8161 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:58:00.741954 8161 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.748325 8161 ts_tablet_manager.cc:1434] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260616 19:58:00.749562 8161 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:00.757184 8161 tablet_bootstrap.cc:654] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:00.760571 8537 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:00.761180 8537 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.779321 8550 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:00.780073 8550 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:00.939958 8537 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:00.945034 8550 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:00.946944 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:00.966420 8563 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:00.984042 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 149277us
I20260616 19:58:00.978598 7867 catalog_manager.cc:5696] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:01.006491 8393 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.030491 8161 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:01.031191 8161 ts_tablet_manager.cc:1403] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.282s user 0.015s sys 0.003s
I20260616 19:58:01.026688 8159 tablet_bootstrap.cc:492] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:01.032044 8159 ts_tablet_manager.cc:1403] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.352s user 0.042s sys 0.114s
I20260616 19:58:00.992940 8563 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.054576 8161 raft_consensus.cc:359] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.055337 8161 raft_consensus.cc:385] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:01.055622 8161 raft_consensus.cc:740] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:58:01.034621 8393 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.058837 8161 consensus_queue.cc:260] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.061698 8160 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:01.062310 8160 ts_tablet_manager.cc:1403] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.357s user 0.074s sys 0.112s
W20260616 19:58:01.082260 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 85576.3us
I20260616 19:58:01.104192 8161 ts_tablet_manager.cc:1434] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.073s user 0.007s sys 0.001s
I20260616 19:58:01.108112 8160 raft_consensus.cc:359] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.111972 8160 raft_consensus.cc:385] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:01.112346 8160 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:01.113159 8160 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.110919 8159 raft_consensus.cc:359] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.116832 8159 raft_consensus.cc:385] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:01.117200 8159 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
W20260616 19:58:01.120615 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 80881.4us
I20260616 19:58:01.127192 8159 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.131453 8160 ts_tablet_manager.cc:1434] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.069s user 0.010s sys 0.000s
I20260616 19:58:01.131960 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:01.144104 8161 tablet_bootstrap.cc:492] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:01.137776 8553 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.146098 8553 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.148689 8393 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:01.150812 8563 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:01.152079 8160 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:01.174353 8159 ts_tablet_manager.cc:1434] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.142s user 0.008s sys 0.001s
I20260616 19:58:01.174340 8549 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:01.203819 8084 leader_election.cc:343] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:01.205436 8159 tablet_bootstrap.cc:492] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:01.197556 8609 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.192641 8549 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.215519 8444 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.248384 8400 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.248925 8400 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.250887 8400 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:01.255192 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:01.227669 8553 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:01.223909 8609 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.277083 8619 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.230346 8444 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.278846 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:01.283699 8160 tablet_bootstrap.cc:654] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:01.295842 8619 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:01.280370 8011 leader_election.cc:343] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:01.301878 8618 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.301941 8609 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:01.302459 8618 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.309389 8618 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:01.310997 8444 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:01.318194 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:01.323791 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:01.324501 7970 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:01.326025 8619 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:01.330935 7967 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:01.337397 8549 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:01.346911 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:01.360333 8161 tablet_bootstrap.cc:654] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:01.366513 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:01.367383 8160 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:01.367806 8160 ts_tablet_manager.cc:1403] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.220s user 0.021s sys 0.013s
I20260616 19:58:01.368950 8112 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:01.379882 8160 raft_consensus.cc:359] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.381753 8160 raft_consensus.cc:385] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:01.382114 8160 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:01.382946 8160 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.388676 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:01.391322 8624 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:01.389537 8119 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:58:01.391862 8624 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.393247 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:01.402366 8160 ts_tablet_manager.cc:1434] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.034s user 0.010s sys 0.000s
I20260616 19:58:01.403017 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:01.403731 8160 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:01.403872 8115 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:01.401965 8624 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:01.454885 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:01.455961 8119 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:01.459151 8159 tablet_bootstrap.cc:654] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:01.466675 8160 tablet_bootstrap.cc:654] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:01.466830 8161 tablet_bootstrap.cc:492] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:01.467237 8161 ts_tablet_manager.cc:1403] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.340s user 0.026s sys 0.033s
I20260616 19:58:01.472561 8159 tablet_bootstrap.cc:492] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:01.473826 8159 ts_tablet_manager.cc:1403] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.284s user 0.008s sys 0.010s
I20260616 19:58:01.471743 8161 raft_consensus.cc:359] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.475484 8161 raft_consensus.cc:385] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:01.476112 8161 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:58:01.477041 8161 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.478647 8159 raft_consensus.cc:359] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.479430 8159 raft_consensus.cc:385] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:01.479741 8159 raft_consensus.cc:740] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:01.480250 8161 ts_tablet_manager.cc:1434] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.013s user 0.009s sys 0.002s
I20260616 19:58:01.480469 8159 consensus_queue.cc:260] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.482882 8159 ts_tablet_manager.cc:1434] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20260616 19:58:01.483253 8160 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:01.483805 8160 ts_tablet_manager.cc:1403] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.080s user 0.016s sys 0.018s
I20260616 19:58:01.483918 8159 tablet_bootstrap.cc:492] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:01.486653 8160 raft_consensus.cc:359] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.487375 8160 raft_consensus.cc:385] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:01.487668 8160 raft_consensus.cc:740] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:01.488413 8160 consensus_queue.cc:260] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:01.490346 8160 ts_tablet_manager.cc:1434] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260616 19:58:01.491441 8160 tablet_bootstrap.cc:492] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:01.498745 8160 tablet_bootstrap.cc:654] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:02.172252 8159 tablet_bootstrap.cc:654] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:02.189733 8159 tablet_bootstrap.cc:492] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:02.190173 8159 ts_tablet_manager.cc:1403] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.707s user 0.010s sys 0.001s
I20260616 19:58:02.187958 8636 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:02.196096 8160 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.695s user 0.058s sys 0.052s
W20260616 19:58:02.196571 8160 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.695s user 0.058s sys 0.053s
I20260616 19:58:02.196663 8636 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.200053 8636 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.200222 8160 tablet_bootstrap.cc:492] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:02.200729 8160 ts_tablet_manager.cc:1403] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.710s user 0.067s sys 0.054s
I20260616 19:58:02.203223 8638 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.203405 8160 raft_consensus.cc:359] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.204118 8160 raft_consensus.cc:385] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:02.204437 8160 raft_consensus.cc:740] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:02.203903 8638 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.205047 8160 consensus_queue.cc:260] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.206610 8638 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.192080 8159 raft_consensus.cc:359] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.207311 8159 raft_consensus.cc:385] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:02.207561 8159 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:02.208271 8159 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.206642 8160 ts_tablet_manager.cc:1434] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260616 19:58:02.211114 8160 tablet_bootstrap.cc:492] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Bootstrap starting.
I20260616 19:58:02.210927 8640 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.211545 8640 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.213699 8640 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.207898 7868 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 86) took 1458 ms. Trace:
I20260616 19:58:02.215394 7868 rpcz_store.cc:276] 0616 19:58:00.748974 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:00.749159 (+ 185us) service_pool.cc:224] Handling call
0616 19:58:02.207717 (+1458558us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6}
I20260616 19:58:02.218575 8159 ts_tablet_manager.cc:1434] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260616 19:58:02.219323 8159 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:02.224483 8160 tablet_bootstrap.cc:654] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:02.225111 8159 tablet_bootstrap.cc:654] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:02.233664 8160 tablet_bootstrap.cc:492] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: No bootstrap required, opened a new log
I20260616 19:58:02.234030 8159 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:02.234117 8160 ts_tablet_manager.cc:1403] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.005s
I20260616 19:58:02.234514 7934 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:02.234689 8159 ts_tablet_manager.cc:1403] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260616 19:58:02.235258 8642 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.235627 8642 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.235968 8642 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.237341 8159 raft_consensus.cc:359] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.237999 8159 raft_consensus.cc:385] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:02.238268 8159 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:02.238914 8159 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.236837 8160 raft_consensus.cc:359] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.241211 8160 raft_consensus.cc:385] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:02.241460 8160 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Initialized, Role: FOLLOWER
I20260616 19:58:02.241873 8159 ts_tablet_manager.cc:1434] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260616 19:58:02.242061 8160 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.242921 8159 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:02.243680 8160 ts_tablet_manager.cc:1434] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260616 19:58:02.246484 8642 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.249053 8642 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.250802 8159 tablet_bootstrap.cc:654] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
W20260616 19:58:02.246963 7967 consensus_meta.cc:287] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.916s user 0.006s sys 0.015s
I20260616 19:58:02.260897 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.261828 8045 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:02.262068 8646 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.262620 8646 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.263768 8637 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.264364 8637 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.266655 8637 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:02.263904 8009 leader_election.cc:343] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:02.267627 8009 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:02.268333 8636 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:02.269407 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.270185 8042 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:02.275591 8645 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.276167 8645 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.278194 8645 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.282691 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.283469 7970 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:02.283864 8159 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:02.284374 8159 ts_tablet_manager.cc:1403] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.010s
I20260616 19:58:02.284768 8649 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.285485 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.286154 8044 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:02.285430 8649 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.287127 8159 raft_consensus.cc:359] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.287842 8159 raft_consensus.cc:385] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:02.288111 8159 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:02.288442 8649 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.288779 8159 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.291132 8159 ts_tablet_manager.cc:1434] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260616 19:58:02.293416 8646 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.296693 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.297396 7966 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
W20260616 19:58:02.301088 8085 leader_election.cc:343] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:02.301637 8085 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.302645 8649 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:02.305794 8646 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.307310 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.306321 8646 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.308841 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.309561 7965 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:02.301246 8159 tablet_bootstrap.cc:492] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:02.309724 8646 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.310854 8161 tablet_bootstrap.cc:492] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:02.311755 8640 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.312340 8640 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.314563 8640 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.315166 8640 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.317725 8650 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:02.322355 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:58:02.328394 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.315672 8640 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.332939 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.334820 8639 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.335449 8641 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.336009 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.336185 8638 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.318117 8161 tablet_bootstrap.cc:654] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:02.428862 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:02.338910 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.339498 8642 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.340246 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:02.340368 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 67529.6us
I20260616 19:58:02.335465 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.371373 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.381996 8084 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:02.379873 8044 raft_consensus.cc:2393] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:02.368515 8040 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 0.
I20260616 19:58:02.366281 8639 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.436859 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.437139 8639 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.437680 8639 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.438134 8639 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.438861 8159 tablet_bootstrap.cc:654] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:02.351351 8650 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.390846 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.393230 8640 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.379660 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 176) took 1008 ms. Trace:
I20260616 19:58:02.378774 8645 consensus_queue.cc:1048] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260616 19:58:02.373469 8655 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.430326 8113 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.370589 8638 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.403784 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.405342 8658 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.449067 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.409116 8660 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.411461 8636 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260616 19:58:02.415598 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.418073 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.454694 7965 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:02.455101 8638 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.455660 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.455765 8660 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.457011 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.419397 8651 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:02.420181 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.424096 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.420032 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:02.431720 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 49842.9us
I20260616 19:58:02.434283 8642 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.369748 8641 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.443146 8650 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.443328 8640 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.443562 8655 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.444504 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.445504 8639 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.445154 8114 rpcz_store.cc:276] 0616 19:58:01.370132 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:01.372605 (+ 2473us) service_pool.cc:224] Handling call
0616 19:58:02.328208 (+955603us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b580019f598. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:58:02.373359 (+ 45151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2854656}
I20260616 19:58:02.477599 8639 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:02.476411 7969 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:02.478367 8655 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.478866 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.432848 8652 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.479725 8652 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.480223 8652 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.481592 7961 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.448819 8658 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.472350 8651 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.485450 8642 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.486096 8658 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.412681 8011 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:02.487157 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 74548.5us
I20260616 19:58:02.487496 8641 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.478078 8639 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.446125 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 174) took 1133 ms. Trace:
I20260616 19:58:02.489379 8640 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:02.492740 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 61706.5us
I20260616 19:58:02.490903 7967 rpcz_store.cc:276] 0616 19:58:01.312712 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:01.314641 (+ 1929us) service_pool.cc:224] Handling call
0616 19:58:01.330676 (+ 16035us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:02.252445 (+921769us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:02.252486 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:02.252925 (+ 439us) log.cc:844] Serialized 26 byte log entry
0616 19:58:02.329486 (+ 76561us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:02.431156 (+101670us) log.cc:844] Serialized 14 byte log entry
0616 19:58:02.441705 (+ 10549us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:02.445858 (+ 4153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:02.445916 (+ 58us) raft_consensus.cc:1685] finished
0616 19:58:02.445927 (+ 11us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:02.446049 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142293,"prepare.run_wall_time_us":2407,"raft.queue_time_us":58635,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":12283,"spinlock_wait_cycles":5410048,"thread_start_us":171516,"threads_started":2,"wal-append.queue_time_us":56918}
I20260616 19:58:02.486162 8651 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.493825 8636 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.494251 8636 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.474668 8084 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:02.496253 7867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 97) took 1236 ms. Trace:
I20260616 19:58:02.496186 8085 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:02.498816 8009 leader_election.cc:343] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:02.492288 7970 raft_consensus.cc:2393] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:02.492172 8639 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.497172 7867 rpcz_store.cc:276] 0616 19:58:01.259463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:01.284221 (+ 24758us) service_pool.cc:224] Handling call
0616 19:58:02.496196 (+1211975us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":225,"reported_tablets":9}
I20260616 19:58:02.494601 8636 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.501487 8650 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260616 19:58:02.501741 8654 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.500943 8660 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.527670 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.513399 8640 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.527326 8654 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.532790 8654 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.545738 8009 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:02.546715 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.548115 8665 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.548524 8665 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.548844 8665 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:58:02.551748 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 53060us
I20260616 19:58:02.550917 8009 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:02.549927 8652 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.556952 8649 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:02.559149 8649 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.560314 8652 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.561232 8649 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:02.557955 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 56740.4us
I20260616 19:58:02.564481 8113 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:02.565718 8651 raft_consensus.cc:2764] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260616 19:58:02.565994 8636 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.567354 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.568063 8636 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.569002 8651 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.578719 8084 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.579429 8646 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.584277 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.585000 8039 raft_consensus.cc:2468] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
W20260616 19:58:02.587533 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 132432us
I20260616 19:58:02.589704 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:02.590377 8161 tablet_bootstrap.cc:492] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:02.590327 8650 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.590902 8650 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.590910 8161 ts_tablet_manager.cc:1403] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.280s user 0.025s sys 0.052s
I20260616 19:58:02.592983 8650 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.593483 8650 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.593782 8650 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.594100 8650 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.594719 8161 raft_consensus.cc:359] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.589768 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.596014 8161 raft_consensus.cc:385] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:02.596509 8161 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
W20260616 19:58:02.596904 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 96686.9us
I20260616 19:58:02.599610 8037 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:02.600605 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.596793 8084 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:02.604377 8161 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.605949 8084 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:02.573906 8654 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.602540 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:02.607546 7961 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.610204 8637 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.612109 8637 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.612407 8668 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.612977 8668 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.613374 8668 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.614198 8637 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.610415 8649 raft_consensus.cc:2804] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.617933 8649 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.618363 8649 raft_consensus.cc:3060] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
W20260616 19:58:02.621510 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 253529us
I20260616 19:58:02.625670 8654 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.627349 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:02.629127 7961 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:02.629313 7935 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.630584 8667 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:02.631932 8637 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:02.633363 8649 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.633842 8637 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.634516 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.635242 8670 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:02.635375 8649 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.636456 8011 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:02.636722 8161 ts_tablet_manager.cc:1434] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.045s user 0.007s sys 0.000s
I20260616 19:58:02.637241 7934 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.638116 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.638305 8655 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.638326 7934 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.625916 8665 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.642242 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.642428 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.652792 8159 tablet_bootstrap.cc:492] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:02.653231 8159 ts_tablet_manager.cc:1403] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.361s user 0.008s sys 0.016s
I20260616 19:58:02.654966 7934 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.655777 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.656114 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.656741 8667 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.657066 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.644230 8085 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.645669 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.664408 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.664960 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.665469 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.666091 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.648101 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.644558 8653 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.647356 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.644812 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.668018 8653 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.668294 7962 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:02.668423 8653 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.645036 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.670876 8645 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.671106 8161 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:02.672254 8665 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.672596 8039 raft_consensus.cc:2393] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:02.646636 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.645730 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:02.676533 8044 raft_consensus.cc:3060] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.665930 8159 raft_consensus.cc:359] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.678051 8159 raft_consensus.cc:385] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:02.678299 8159 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:02.673206 8085 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.644932 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.685680 8159 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.687618 8656 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:02.688097 8654 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.685526 8645 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:02.687709 8637 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.688231 8656 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.693893 8656 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.694278 8645 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.694667 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:02.697647 8085 leader_election.cc:343] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:02.699230 7935 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:02.700177 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45220us
I20260616 19:58:02.707311 8159 ts_tablet_manager.cc:1434] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.054s user 0.006s sys 0.001s
I20260616 19:58:02.710748 7935 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.702443 8655 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election won for term 1
W20260616 19:58:02.713042 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 74549.7us
I20260616 19:58:02.713428 8159 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:02.644565 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:02.717633 8661 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.717823 8654 consensus_queue.cc:1048] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:02.696233 8650 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.692692 8668 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.723062 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.732196 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:02.730001 8011 leader_election.cc:343] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:02.720896 8639 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.732535 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.735375 8011 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:02.738905 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.740971 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.743207 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.743600 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.744333 7934 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.722153 8655 raft_consensus.cc:697] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:02.747126 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.744858 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.754737 7934 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.752049 8118 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:02.759691 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:02.771665 8664 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.758038 7934 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.756718 8665 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.769090 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:02.776669 8037 raft_consensus.cc:2393] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:58:02.777714 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 69131.8us
W20260616 19:58:02.801362 8085 leader_election.cc:343] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:02.802636 8085 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.768551 8645 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.768235 8640 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.777576 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.756002 8639 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.757555 8656 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.775967 8637 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:02.775733 8667 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.831878 8161 tablet_bootstrap.cc:654] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
W20260616 19:58:02.833657 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 56181.9us
I20260616 19:58:02.807920 8044 raft_consensus.cc:2468] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:02.818147 8637 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:02.849280 8637 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.809149 8665 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.828924 8664 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:02.827484 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.821142 8668 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:02.834901 8645 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:02.817849 8656 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.857856 8640 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:02.865562 8658 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.913455 8682 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.913723 8679 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.914263 8682 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.914399 8679 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.914914 8689 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.915794 8689 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:02.933269 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 112420us
I20260616 19:58:02.936343 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.936903 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.937686 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.939903 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:02.926416 8682 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.890532 8665 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.819298 8655 consensus_queue.cc:237] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.949077 7934 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:02.934612 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:02.908646 8640 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.916989 8645 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:02.962276 8683 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.917284 8658 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.949662 8650 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.857076 8664 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.969123 8636 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.952754 8660 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:02.954232 8640 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:02.962806 8683 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.980439 8689 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:02.957587 8692 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.953461 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:02.959075 7963 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:02.988427 8686 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:02.988270 8636 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:02.992646 8677 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.998842 8159 tablet_bootstrap.cc:654] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:02.935572 8653 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.001940 8675 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:58:03.001538 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 52559.3us
I20260616 19:58:02.992105 8654 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:02.999224 8692 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.002511 8675 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.017995 8084 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:03.005456 7934 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:03.008575 8677 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.023272 8662 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.008800 8679 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.017880 8649 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.025592 8680 raft_consensus.cc:2804] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:03.024325 7934 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:03.028218 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.029191 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.031836 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:03.012634 8654 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.017115 8637 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.025748 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:03.067425 8011 leader_election.cc:343] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:03.003754 8645 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.068291 7934 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:03.064213 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.036947 8679 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:03.086985 8693 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.085973 8699 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.038049 8662 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.003971 8686 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.037748 8649 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.092939 8670 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.088114 8680 raft_consensus.cc:697] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:03.088315 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.103257 8641 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.142421 8638 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.072479 7963 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:03.203410 8161 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:03.204093 8161 ts_tablet_manager.cc:1403] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.542s user 0.038s sys 0.032s
I20260616 19:58:03.151708 8693 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.233594 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.234112 8675 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.236248 8690 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.147270 8679 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.269160 8686 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.275344 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.277580 8011 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:03.205861 8638 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.153446 8699 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.291514 8682 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:03.237061 8690 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.252614 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:03.297498 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 229268us
I20260616 19:58:03.308926 8161 raft_consensus.cc:359] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.309855 8161 raft_consensus.cc:385] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:03.310153 8161 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:58:03.310989 8161 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.254778 8692 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.314971 7934 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:03.316992 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:03.317698 7966 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:03.317602 8688 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.284742 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.316864 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.323374 8637 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.330163 8654 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.291894 8680 consensus_queue.cc:237] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.333544 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.301640 8683 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.308401 8682 raft_consensus.cc:697] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:03.381220 8161 ts_tablet_manager.cc:1434] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.177s user 0.006s sys 0.002s
I20260616 19:58:03.384347 8116 raft_consensus.cc:2393] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:03.326319 8692 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:03.333600 8140 debug-util.cc:398] Leaking SignalData structure 0x7b0800085be0 after lost signal to thread 6785
I20260616 19:58:03.347657 8642 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.335711 8115 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:03.386157 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:03.389381 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.401535 8116 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 0.
I20260616 19:58:03.390923 8693 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.382428 8688 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.419144 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:03.420739 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.424952 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:03.419214 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.427266 8114 raft_consensus.cc:2393] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:03.428129 8699 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.424921 8690 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.443368 8679 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.447995 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.414386 7968 raft_consensus.cc:2393] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 1: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:03.450287 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:03.417527 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08001332c0 after lost signal to thread 7902
I20260616 19:58:03.398487 8642 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.415148 8638 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.412596 8682 consensus_queue.cc:237] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.439273 8640 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.476199 8677 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.464797 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:03.478572 7969 raft_consensus.cc:2393] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:03.478945 8642 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.479488 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:03.480646 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.482633 8161 tablet_bootstrap.cc:492] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b: Bootstrap starting.
I20260616 19:58:03.483282 7961 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:03.468564 8702 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:03.485303 8640 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:03.485731 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08002dda40 after lost signal to thread 7994
I20260616 19:58:03.492317 7966 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
W20260616 19:58:03.492695 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08002de280 after lost signal to thread 8069
W20260616 19:58:03.494035 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1140 after lost signal to thread 8143
I20260616 19:58:03.488178 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:03.485152 8702 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.502138 7968 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:03.503918 8159 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:03.504469 8159 ts_tablet_manager.cc:1403] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.794s user 0.008s sys 0.034s
I20260616 19:58:03.507332 8159 raft_consensus.cc:359] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.508042 8159 raft_consensus.cc:385] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:03.512729 8159 raft_consensus.cc:740] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:03.514947 8159 consensus_queue.cc:260] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.543147 8159 ts_tablet_manager.cc:1434] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.038s user 0.005s sys 0.002s
I20260616 19:58:03.545105 8658 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.549782 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.551651 7968 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:03.547539 8688 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.554872 8702 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.569020 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.570470 7961 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:03.571983 8085 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:03.573781 8085 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:03.575196 8085 leader_election.cc:343] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:03.573726 8645 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:03.575714 8652 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.578786 8645 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.574155 8646 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:58:03.580520 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.581283 8645 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.579981 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.581915 8085 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:03.582647 8645 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:03.583024 8645 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.583402 8645 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:03.584405 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.590163 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.586467 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.585518 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.591188 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.597885 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.599021 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:03.599582 7963 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:03.601725 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:03.602276 7970 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:03.604395 8664 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.607226 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.607870 8113 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:03.608289 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:03.607888 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.612129 8110 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:03.613205 8653 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.613917 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.615020 8161 tablet_bootstrap.cc:654] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:03.617141 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:03.618253 8653 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:03.619045 8653 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:03.619347 8673 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:03.619445 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:03.619809 8673 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:03.622848 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.624038 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:03.625140 7916 auto_rebalancer.cc:259] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260616 19:58:03.626466 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.620113 8039 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:03.628783 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:03.629364 8110 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:03.634502 8656 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.639668 8039 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:03.640578 8636 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.641280 7970 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:03.643513 8649 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.640775 8159 tablet_bootstrap.cc:492] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:03.645799 8009 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:03.646900 8009 leader_election.cc:343] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:03.648270 8684 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.648367 7934 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:03.652591 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.653853 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.656131 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:03.657889 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:03.647830 8009 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:03.683756 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.619478 8653 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:03.685868 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.686182 8653 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:03.692912 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45153.1us
I20260616 19:58:03.693922 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:03.697114 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 48847.3us
I20260616 19:58:03.698457 8085 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:03.701575 8009 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:03.703233 8009 leader_election.cc:343] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:03.705557 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.706310 8085 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:03.707262 8043 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:03.707137 8009 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:03.719786 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.706250 8639 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:03.720988 8639 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:03.720368 7868 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:03.704658 8636 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.704414 8679 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.700532 8116 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:03.704013 8664 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.721376 8639 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:03.708711 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:03.736330 8645 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.741463 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.715144 8641 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260616 19:58:03.714679 8656 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:03.712762 8680 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:03.730729 8673 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.785002 8673 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:03.781200 7966 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:03.782177 8159 tablet_bootstrap.cc:654] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:03.731469 8009 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:03.790234 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 57986.6us
W20260616 19:58:03.790778 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 59375.6us
I20260616 19:58:03.791441 8009 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:03.802029 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.803556 7867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 98) took 1087 ms. Trace:
I20260616 19:58:03.803985 7867 rpcz_store.cc:276] 0616 19:58:02.716586 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:02.718042 (+ 1456us) service_pool.cc:224] Handling call
0616 19:58:03.803498 (+1085456us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":16}
I20260616 19:58:03.808588 8009 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:03.789249 8645 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:03.810596 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:03.797376 8680 raft_consensus.cc:697] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:03.803687 8679 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:03.817126 8011 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:03.818909 8009 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:03.815591 8653 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:03.820798 8653 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:03.821201 8653 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:03.823249 8161 tablet_bootstrap.cc:492] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b: No bootstrap required, opened a new log
I20260616 19:58:03.823732 8161 ts_tablet_manager.cc:1403] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b: Time spent bootstrapping tablet: real 0.341s user 0.032s sys 0.037s
I20260616 19:58:03.819605 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.827180 8653 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.829034 8653 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:03.833261 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:03.830022 8637 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:58:03.838596 8685 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:03.838989 8009 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:03.839298 8685 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.841496 8682 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260616 19:58:03.842777 8685 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.845806 8691 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:03.833774 8664 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:03.819916 8679 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:03.849767 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.837642 8689 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:03.852780 8689 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.855098 8689 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.860363 8680 consensus_queue.cc:237] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.847749 8691 raft_consensus.cc:697] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:03.837934 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:03.837092 8116 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:03.865772 8691 consensus_queue.cc:237] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.851094 8679 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:03.868739 8675 raft_consensus.cc:2764] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260616 19:58:03.872176 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:04.002044 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 130375us
I20260616 19:58:03.986110 8639 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:03.997589 8679 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.008787 8639 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:03.970654 8110 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:04.010617 8679 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:04.002569 8159 tablet_bootstrap.cc:492] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:03.901413 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:04.011528 8159 ts_tablet_manager.cc:1403] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.371s user 0.014s sys 0.003s
I20260616 19:58:03.895606 8044 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:04.014317 8159 raft_consensus.cc:359] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.014986 8159 raft_consensus.cc:385] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:03.899967 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:04.015391 8159 raft_consensus.cc:740] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:04.007985 8009 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.017350 8009 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.021065 8701 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:04.022027 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:04.022583 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:04.021533 8701 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.033219 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:04.033473 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:04.035871 8653 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.035899 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:04.036644 8653 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.036939 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:04.037268 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:04.038048 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:04.038877 8009 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.040345 8718 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.041196 7963 raft_consensus.cc:3060] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:04.045804 7970 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:04.048820 8701 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:04.016036 8159 consensus_queue.cc:260] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.051597 8159 ts_tablet_manager.cc:1434] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
I20260616 19:58:04.059197 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:04.061182 8009 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.063453 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:04.064289 8161 raft_consensus.cc:359] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:04.064957 8011 leader_election.cc:343] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Illegal state: must be running to vote when last-logged opid is not known
I20260616 19:58:04.067390 8084 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.068519 8686 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.070197 8718 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.070855 8159 tablet_bootstrap.cc:492] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Bootstrap starting.
I20260616 19:58:04.071715 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.072299 8668 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.072857 8699 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.077064 8084 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.078933 7935 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.078997 8699 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.080013 8084 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.080761 8699 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.081946 8084 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.082741 8699 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.086369 8691 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.086611 8682 consensus_queue.cc:1059] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:04.087697 8691 raft_consensus.cc:3055] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:04.087790 8009 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.088552 8718 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.088088 8691 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:04.089238 8084 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.089407 8691 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.090041 8699 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.090621 8691 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:04.064976 8161 raft_consensus.cc:385] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:04.091367 8161 raft_consensus.cc:740] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Initialized, Role: FOLLOWER
I20260616 19:58:04.091953 8161 consensus_queue.cc:260] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.092298 8084 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.093652 8645 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.095142 8084 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.095953 8699 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.097146 8668 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.102874 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:04.104130 8084 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.094069 8161 ts_tablet_manager.cc:1434] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b: Time spent starting tablet: real 0.270s user 0.016s sys 0.008s
I20260616 19:58:04.112252 8117 raft_consensus.cc:1240] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:04.112458 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:04.106794 8668 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.113648 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:04.114260 8112 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:04.116472 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:04.119536 8043 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:04.120325 8009 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.109973 8688 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.121266 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:04.122153 8040 raft_consensus.cc:2393] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 1: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:04.122058 8699 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:04.125663 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:04.128228 8009 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.124848 8718 consensus_queue.cc:1059] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:04.132650 7935 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.133600 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.129252 8679 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.137431 8655 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.137830 8661 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.139086 8716 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.139173 8718 raft_consensus.cc:3055] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:04.139183 7935 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.140926 7935 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.139740 8718 raft_consensus.cc:740] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:04.141721 8661 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.142697 8639 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:04.144795 7935 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:04.148561 8009 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.142593 8718 consensus_queue.cc:260] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:04.193764 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45411us
I20260616 19:58:04.198277 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:04.149632 8661 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:04.129601 8680 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260616 19:58:04.200774 7867 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:04.202769 7935 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:04.148420 8701 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260616 19:58:04.206301 8692 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:04.203744 7867 catalog_manager.cc:5696] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:04.212319 8655 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260616 19:58:04.212949 8661 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:04.215478 8159 tablet_bootstrap.cc:654] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Neither blocks nor log segments found. Creating new log.
I20260616 19:58:04.195187 8716 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:04.217417 8718 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:04.225070 8692 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:04.213774 8661 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.230317 8692 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.239573 8112 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:04.230242 8717 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260616 19:58:04.274983 8009 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:04.276998 8691 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:04.265853 8675 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:04.279239 8691 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:04.282620 8675 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.283309 8691 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.284708 8675 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:04.296317 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:04.298866 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:04.307003 7963 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
I20260616 19:58:04.307363 8036 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 0.
W20260616 19:58:04.308470 8722 log.cc:927] Time spent T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Append to log took a long time: real 0.079s user 0.001s sys 0.000s
I20260616 19:58:04.308782 8084 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:04.329438 8724 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.091s user 0.001s sys 0.000s
I20260616 19:58:04.329478 8159 tablet_bootstrap.cc:492] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: No bootstrap required, opened a new log
I20260616 19:58:04.330130 8159 ts_tablet_manager.cc:1403] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Time spent bootstrapping tablet: real 0.259s user 0.023s sys 0.003s
I20260616 19:58:04.341670 8651 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260616 19:58:04.347869 8651 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:04.348230 8651 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:04.376747 8159 raft_consensus.cc:359] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.385843 8159 raft_consensus.cc:385] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260616 19:58:04.386240 8159 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Initialized, Role: FOLLOWER
I20260616 19:58:04.386968 8159 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.379699 8637 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.391456 8159 ts_tablet_manager.cc:1434] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Time spent starting tablet: real 0.061s user 0.006s sys 0.000s
I20260616 19:58:04.402645 8651 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.421452 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:04.422282 8036 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:04.421674 8651 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:04.424012 8679 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260616 19:58:04.425601 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:04.426574 7963 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:04.442930 8040 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:04.448045 8667 consensus_queue.cc:1059] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:04.448812 7963 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:04.449334 8667 raft_consensus.cc:3055] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:04.449787 8667 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:04.450567 8085 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:04.451411 8652 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader election won for term 1
I20260616 19:58:04.451937 8652 raft_consensus.cc:697] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:04.451506 8667 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.452893 8652 consensus_queue.cc:237] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:04.453294 8667 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:04.468911 8716 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.469113 8036 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:04.478709 8706 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.484306 8686 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.497105 7866 catalog_manager.cc:5696] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 1 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:04.515380 7867 catalog_manager.cc:5696] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:04.516994 7867 catalog_manager.cc:5696] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:04.520741 7867 catalog_manager.cc:5696] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:04.540905 8716 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.560026 7868 catalog_manager.cc:5696] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:04.571143 7868 catalog_manager.cc:5696] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:04.591775 7866 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:04.594806 7866 catalog_manager.cc:5696] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:04.633987 8036 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:04.635661 8667 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:04.654531 8112 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:04.661126 8112 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:04.662686 8667 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.678587 8716 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.698899 7970 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 0 FOLLOWER]: Advancing to term 1
I20260616 19:58:04.730847 8716 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.838157 8036 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:04.839627 8667 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:04.861001 8112 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:04.880749 8112 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:04.884414 8661 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:04.978308 8651 consensus_queue.cc:1048] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:04.991353 8651 consensus_queue.cc:1048] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260616 19:58:05.777083 8651 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:05.777627 8651 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:05.779661 8651 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:05.780313 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:05.780930 7963 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:05.781483 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:05.782121 8044 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:05.782198 8085 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:05.783130 8651 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:05.783586 8651 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:05.783929 8651 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:05.791435 8651 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:05.794234 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:05.794795 7963 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:05.795629 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:05.796132 8044 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:05.802237 7963 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:05.803426 8085 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:05.804231 8706 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:05.804692 8706 raft_consensus.cc:697] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:05.805542 8706 consensus_queue.cc:237] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:05.793663 8651 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:05.821547 8044 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:05.825539 7867 catalog_manager.cc:5696] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:05.831084 8651 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:05.831596 8651 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:05.833431 8651 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:05.837743 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:05.838429 7970 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:05.842288 8085 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:05.840735 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:05.843328 8651 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:05.861733 8651 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:05.862107 8651 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:05.843603 8044 raft_consensus.cc:2393] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:05.868796 8651 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:05.871088 8651 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:05.872624 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:05.876931 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:05.877733 8040 raft_consensus.cc:2393] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:05.881258 7963 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:05.882911 8085 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:05.884021 8706 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:05.884760 8706 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:05.885591 8706 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:05.903088 7867 catalog_manager.cc:5696] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:06.300215 8691 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:06.300798 8691 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:06.303396 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:06.304123 7970 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:06.304526 8691 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:06.304739 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:06.305333 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:06.306345 8691 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:06.306821 8691 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:06.307199 8691 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:06.312079 8691 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:06.313643 8691 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:06.314745 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:06.314805 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:06.317162 7970 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:06.322824 7970 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:06.323829 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:06.324762 8691 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:06.325295 8691 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:06.326201 8691 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:06.351559 7867 catalog_manager.cc:5696] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 2 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:06.358021 7963 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:06.360769 8651 consensus_queue.cc:1059] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:06.361541 8651 raft_consensus.cc:3055] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:06.361840 8651 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:06.362560 8651 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:06.363603 8651 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:06.378156 6782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260616 19:58:06.396788 8764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260616 19:58:06.410235 8766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:58:06.410421 7965 raft_consensus.cc:1275] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:06.413549 8706 consensus_queue.cc:1048] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260616 19:58:06.414058 8763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260616 19:58:06.420464 6782 server_base.cc:1061] running on GCE node
I20260616 19:58:06.421983 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260616 19:58:06.422452 6782 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260616 19:58:06.422768 6782 hybrid_clock.cc:648] HybridClock initialized: now 1781639886422741 us; error 0 us; skew 500 ppm
I20260616 19:58:06.423743 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260616 19:58:06.432503 8040 raft_consensus.cc:1275] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:06.434239 8686 consensus_queue.cc:1048] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:06.465498 6782 webserver.cc:533] Webserver started at http://127.6.159.132:35017/ using document root <none> and password file <none>
I20260616 19:58:06.466645 6782 fs_manager.cc:362] Metadata directory not provided
I20260616 19:58:06.467042 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260616 19:58:06.467545 6782 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260616 19:58:06.469769 6782 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-3-root/instance:
uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
format_stamp: "Formatted at 2026-06-16 19:58:06 on dist-test-slave-1c0m"
I20260616 19:58:06.609527 6782 fs_manager.cc:696] Time spent creating directory manager: real 0.139s user 0.026s sys 0.041s
I20260616 19:58:06.643522 8782 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260616 19:58:06.651329 6782 fs_manager.cc:730] Time spent opening block manager: real 0.034s user 0.006s sys 0.008s
I20260616 19:58:06.651769 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-3-root
uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
format_stamp: "Formatted at 2026-06-16 19:58:06 on dist-test-slave-1c0m"
I20260616 19:58:06.652180 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260616 19:58:06.692286 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260616 19:58:06.694131 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260616 19:58:06.696126 6782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260616 19:58:06.699128 6782 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260616 19:58:06.699314 6782 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260616 19:58:06.699625 6782 ts_tablet_manager.cc:616] Registered 0 tablets
I20260616 19:58:06.699791 6782 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260616 19:58:06.780386 8110 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260616 19:58:06.780961 8110 pending_rounds.cc:85] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 1
I20260616 19:58:06.781451 8110 pending_rounds.cc:107] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260616 19:58:06.782146 8110 raft_consensus.cc:2889] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:06.784624 8776 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:06.796077 7965 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:06.804399 8777 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:06.859012 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.159.132:37343
I20260616 19:58:06.859102 8845 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.159.132:37343 every 8 connection(s)
I20260616 19:58:06.908926 6782 auto_leader_rebalancer-test.cc:439] add a tserver: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:06.943792 8846 heartbeater.cc:344] Connected to a master server at 127.6.159.190:44129
I20260616 19:58:06.944353 8846 heartbeater.cc:461] Registering TS with master...
I20260616 19:58:06.950573 8846 heartbeater.cc:507] Master 127.6.159.190:44129 requested a full tablet report, sending...
I20260616 19:58:06.953567 7867 ts_manager.cc:194] Registered new tserver with Master: c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:06.955711 7867 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59082
I20260616 19:58:07.958813 8846 heartbeater.cc:499] Master 127.6.159.190:44129 was elected leader, sending a full tablet report...
I20260616 19:58:08.095003 8110 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:08.103253 8041 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.105489 7967 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.106861 8856 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:08.109210 8774 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260616 19:58:08.120846 8686 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.126669 7967 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.128554 8041 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.135776 8110 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:08.142547 7867 catalog_manager.cc:5696] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260616 19:58:08.146297 8082 consensus_peers.cc:597] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5f64cb36408849c989ecda8d19ec4269. This is attempt 1: this message will repeat every 5th retry.
W20260616 19:58:08.150239 8082 consensus_peers.cc:597] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd3a274b9cd54eea9bdf95a6e2c9e13a. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:08.152846 7963 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.155654 8044 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.156018 8856 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:08.159612 8856 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:08.179661 8686 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.194902 7967 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.227571 8040 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:08.236799 8044 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.249639 7867 catalog_manager.cc:5696] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:08.273782 8110 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.275277 8777 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:08.296489 7967 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.298730 8777 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:08.309019 8777 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.337360 8112 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.345434 7967 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.371169 7868 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 1 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
W20260616 19:58:08.381091 8008 consensus_peers.cc:597] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf8a24d3c225486fa9270926ffe6a40e. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:08.371353 8112 consensus_queue.cc:237] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:58:08.407415 8082 consensus_peers.cc:597] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bf8afc79e3c417c814ac2fb57557096. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:08.409502 7967 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.412505 8041 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260616 19:58:08.411398 8856 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260616 19:58:08.413796 8774 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:08.425632 8774 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.430104 8041 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.430204 7967 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.458154 7868 catalog_manager.cc:5696] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index -1 to 2, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:08.621409 8879 ts_tablet_manager.cc:933] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:08.637069 8879 tablet_copy_client.cc:323] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.131:40887
I20260616 19:58:08.663815 8129 tablet_copy_service.cc:140] P 6827184e3cef42c2ada15b2d31afee4b: Received BeginTabletCopySession request for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:34110)
I20260616 19:58:08.664548 8129 tablet_copy_service.cc:161] P 6827184e3cef42c2ada15b2d31afee4b: Beginning new tablet copy session on tablet fd3a274b9cd54eea9bdf95a6e2c9e13a from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:34110: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-fd3a274b9cd54eea9bdf95a6e2c9e13a
I20260616 19:58:08.672804 8129 tablet_copy_source_session.cc:215] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:08.677647 8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:08.691340 8883 ts_tablet_manager.cc:933] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:08.728905 8879 tablet_copy_client.cc:806] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:08.729737 8879 tablet_copy_client.cc:670] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:08.734365 8883 tablet_copy_client.cc:323] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.131:40887
I20260616 19:58:08.737368 8879 tablet_copy_client.cc:538] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:08.737468 8129 tablet_copy_service.cc:140] P 6827184e3cef42c2ada15b2d31afee4b: Received BeginTabletCopySession request for tablet 5f64cb36408849c989ecda8d19ec4269 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:34110)
I20260616 19:58:08.738039 8129 tablet_copy_service.cc:161] P 6827184e3cef42c2ada15b2d31afee4b: Beginning new tablet copy session on tablet 5f64cb36408849c989ecda8d19ec4269 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:34110: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-5f64cb36408849c989ecda8d19ec4269
I20260616 19:58:08.745353 8129 tablet_copy_source_session.cc:215] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:08.748833 8883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f64cb36408849c989ecda8d19ec4269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:08.760351 8879 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:08.768781 8883 tablet_copy_client.cc:806] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:08.769577 8883 tablet_copy_client.cc:670] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:08.775637 8883 tablet_copy_client.cc:538] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:08.787668 8883 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:08.819905 8883 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:08.821100 8883 tablet_bootstrap.cc:492] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:08.821882 8883 ts_tablet_manager.cc:1403] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.010s
I20260616 19:58:08.825441 8879 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:08.825495 8883 raft_consensus.cc:359] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:08.826574 8883 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:08.826964 8879 tablet_bootstrap.cc:492] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:08.828927 8879 ts_tablet_manager.cc:1403] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.069s user 0.020s sys 0.011s
I20260616 19:58:08.829289 8883 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:08.836143 8879 raft_consensus.cc:359] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:08.837039 8879 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:08.837685 8879 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:08.861510 8883 ts_tablet_manager.cc:1434] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.039s user 0.005s sys 0.003s
I20260616 19:58:08.898670 8129 tablet_copy_service.cc:342] P 6827184e3cef42c2ada15b2d31afee4b: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-5f64cb36408849c989ecda8d19ec4269 received from {username='slave'} at 127.0.0.1:34110
I20260616 19:58:08.899262 8129 tablet_copy_service.cc:434] P 6827184e3cef42c2ada15b2d31afee4b: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-5f64cb36408849c989ecda8d19ec4269 on tablet 5f64cb36408849c989ecda8d19ec4269 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:08.912359 8883 ts_tablet_manager.cc:933] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:08.917745 8883 tablet_copy_client.cc:323] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.130:43051
I20260616 19:58:08.983873 8888 ts_tablet_manager.cc:933] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:09.014748 8888 tablet_copy_client.cc:323] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.131:40887
I20260616 19:58:09.031726 8129 tablet_copy_service.cc:140] P 6827184e3cef42c2ada15b2d31afee4b: Received BeginTabletCopySession request for tablet 2bf8afc79e3c417c814ac2fb57557096 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:34110)
I20260616 19:58:09.032528 8129 tablet_copy_service.cc:161] P 6827184e3cef42c2ada15b2d31afee4b: Beginning new tablet copy session on tablet 2bf8afc79e3c417c814ac2fb57557096 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:34110: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-2bf8afc79e3c417c814ac2fb57557096
I20260616 19:58:09.094550 8879 ts_tablet_manager.cc:1434] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.264s user 0.006s sys 0.000s
I20260616 19:58:09.100679 8128 tablet_copy_service.cc:342] P 6827184e3cef42c2ada15b2d31afee4b: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-fd3a274b9cd54eea9bdf95a6e2c9e13a received from {username='slave'} at 127.0.0.1:34110
I20260616 19:58:09.101248 8128 tablet_copy_service.cc:434] P 6827184e3cef42c2ada15b2d31afee4b: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-fd3a274b9cd54eea9bdf95a6e2c9e13a on tablet fd3a274b9cd54eea9bdf95a6e2c9e13a with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:09.158342 8820 raft_consensus.cc:1217] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260616 19:58:09.186919 8129 tablet_copy_source_session.cc:215] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:09.899273 8891 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:09.900198 8891 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:09.904796 8891 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:09.923169 8891 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:09.923691 8891 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:09.925745 8891 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:09.933394 8894 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:09.933980 8894 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.024698 8777 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.025295 8777 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.028302 8777 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:10.402081 6787 debug-util.cc:398] Leaking SignalData structure 0x7b080000f860 after lost signal to thread 8009
I20260616 19:58:09.946506 8894 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:10.414463 6787 kernel_stack_watchdog.cc:198] Thread 8009 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 345ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260616 19:58:10.402535 8777 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.416112 8777 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:10.439814 8008 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.032s sys 0.067s
I20260616 19:58:10.406380 8891 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:58:10.440687 8008 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s user 0.033s sys 0.067s
I20260616 19:58:10.451443 8896 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.451977 8896 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.449312 8895 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.449957 8899 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.451864 8898 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.452610 8897 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.452656 8902 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.445284 8891 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.460331 8898 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.462749 8901 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.472806 8901 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.474613 8898 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.474931 8901 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.475117 8898 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.475318 8901 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.475669 8901 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.477176 8901 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:10.477303 8085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.511s user 0.013s sys 0.039s
W20260616 19:58:10.477630 8085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.511s user 0.013s sys 0.039s
I20260616 19:58:10.458616 8895 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.481876 8777 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.482425 8777 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.482920 8777 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.483778 8895 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.475590 8898 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.487805 8777 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.490191 8777 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.491158 8777 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.493906 8891 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.461241 8899 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.494426 8891 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.494776 8686 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.494949 8891 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.495302 8686 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.460947 8897 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.496611 8777 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.497293 8777 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.498039 8897 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.497859 8777 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.470329 8896 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.500365 8898 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.501715 8891 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.501822 8777 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.504490 8686 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.462463 8902 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.509491 8902 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.539400 8907 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.541347 8907 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.544854 8908 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.546396 8907 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.547775 8907 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.547753 8908 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.548938 8907 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:10.552054 8908 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.552351 8907 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.554970 8908 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.556350 8907 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.556563 8908 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.557463 8907 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.560976 8908 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.563911 8908 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.553815 8914 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.565740 8908 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.562284 8907 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.565939 8914 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.564731 8913 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.569685 8917 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:10.578564 8009 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 1.657s user 0.022s sys 0.001s
W20260616 19:58:10.580077 8009 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 1.659s user 0.023s sys 0.002s
I20260616 19:58:10.581307 8914 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.581570 8907 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.581979 8914 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.582367 8907 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.582789 8914 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.590395 8913 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.596890 8777 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.597911 8777 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.596544 8686 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.602243 8895 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.602396 8891 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.603313 8895 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:10.603394 8891 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.604813 8686 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.607084 8686 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.607157 8895 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.607259 8891 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.613919 8907 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.614794 8913 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.611781 8894 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.616828 8777 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.581274 8908 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.616845 8895 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.624082 8894 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.624641 8895 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.627070 8894 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.620218 8908 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.620963 8777 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.621603 8893 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.629735 8908 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.630048 8917 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.627365 8895 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.628765 8914 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.629408 8894 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.630172 8893 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.623411 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf8afc79e3c417c814ac2fb57557096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:10.619122 8907 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.629776 8777 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.632892 8908 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.633391 8917 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.642112 8898 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.651588 8898 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.653591 8777 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.653817 8898 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.654150 8777 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.654845 8777 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.642867 8903 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.574481 8911 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.643988 8901 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.644717 8896 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.645228 8897 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.631422 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 5) took 1720 ms. Trace:
I20260616 19:58:10.646121 8894 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.639133 8895 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.659212 8897 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.660269 8896 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.660853 8903 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.664719 8901 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:10.658002 7866 rpcz_store.cc:276] 0616 19:58:08.910855 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:08.911002 (+ 147us) service_pool.cc:224] Handling call
0616 19:58:10.631199 (+1720197us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260616 19:58:10.673583 8897 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.674381 8893 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.667665 8907 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.681325 8914 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.682108 8914 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.706264 8911 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.709661 8895 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.684063 8917 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.713236 8820 raft_consensus.cc:1217] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260616 19:58:10.714031 8917 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:10.713295 8908 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.717166 8908 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.719676 8686 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.715770 8895 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.719930 8908 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.783308 8904 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.781774 8906 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.889504 8906 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.864076 8895 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.888584 8902 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.892297 8686 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.892925 8920 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.764906 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:10.896705 8894 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.757838 8913 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.750427 8891 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.854741 8917 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.895813 8903 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.853613 8896 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.782588 8905 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.908664 8921 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:10.912765 8891 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.908185 8777 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.915907 8893 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.932783 8893 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.921314 8907 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.920992 8904 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.921815 8920 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.923628 8915 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.924136 8905 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.924240 8902 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.925006 8914 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.938562 8921 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.915365 8895 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.908134 8111 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:10.960726 8915 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.929386 8911 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.975732 8907 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.976924 8917 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:10.977248 8686 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.792205 8899 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:10.977967 8917 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.915190 8913 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.987303 8891 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:10.978664 8908 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.988583 8908 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.992380 8922 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:10.993135 8922 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.994040 8907 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:10.999069 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.000232 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.001184 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.002103 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.002930 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.003567 8117 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:11.015280 8898 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.016245 8898 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.011698 8908 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.017438 8908 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.018025 8908 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.019212 8918 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.019946 8918 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.015383 8916 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.022724 8905 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.026990 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.027276 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.029166 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.029402 8906 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.029925 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.031692 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.038518 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.039182 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.015226 8896 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.030476 8894 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.036818 8117 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.045055 8895 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.049901 8112 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.053628 8896 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.052523 8905 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.051229 8897 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.057706 8905 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.058996 8896 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.059741 8894 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.060600 8910 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.061224 8910 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.062140 8894 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.063657 8910 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.050259 8893 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.044739 8916 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.065117 8902 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.058574 8897 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:11.048947 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.069149 8901 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.069312 8916 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.070052 8908 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.048738 8906 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.045495 8899 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.052093 8904 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.072918 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.116858 8116 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.120290 8910 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.109586 8911 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.120611 8897 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.121246 8907 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.114543 8923 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.125785 8913 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.126506 8918 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.126672 8917 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.122244 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.124236 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.049533 8888 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52458 (request call id 5) took 2055 ms. Trace:
I20260616 19:58:11.071468 8921 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.126415 8923 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.129211 8922 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.125778 8911 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.124975 8910 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.129700 8777 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.132815 8777 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.115504 8905 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.134948 8777 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.135334 8897 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.115267 8898 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.135941 8897 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.136264 8913 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.136416 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.137140 8119 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.137593 8899 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.136912 8913 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.138175 8897 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.140751 8909 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.141877 8923 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.141846 8909 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.149042 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.078956 8920 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.150761 8913 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.131518 8110 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.139814 8915 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.152305 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.153071 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.154860 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.157146 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.157675 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.158169 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.158602 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.176469 8909 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.160677 8903 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.160343 8897 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.181468 8903 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.149726 8906 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.183051 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.183229 8119 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.184132 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.184406 8906 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.160621 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.184775 8903 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.181854 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.160477 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.188561 8117 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.130478 8888 rpcz_store.cc:276] 0616 19:58:08.982771 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:08.982976 (+ 205us) service_pool.cc:224] Handling call
0616 19:58:08.984325 (+ 1349us) ts_tablet_manager.cc:934] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
0616 19:58:10.620711 (+1636386us) tablet_copy_client.cc:368] Tablet copy session begun
0616 19:58:10.784902 (+164191us) tablet_metadata.cc:665] Metadata flushed
0616 19:58:10.785074 (+ 172us) tablet_copy_client.cc:497] Wrote new tablet metadata
0616 19:58:11.020131 (+235057us) tablet_copy_client.cc:418] Wrote new consensus metadata
0616 19:58:11.020536 (+ 405us) tablet_replica.cc:206] Creating consensus instance
0616 19:58:11.028071 (+ 7535us) ts_tablet_manager.cc:989] Replica registered.
0616 19:58:11.037123 (+ 9052us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":301,"tablet-copy.queue_time_us":130,"thread_start_us":29316,"threads_started":1}
I20260616 19:58:11.176491 8911 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.189826 8888 tablet_copy_client.cc:806] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:11.190505 8888 tablet_copy_client.cc:670] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:11.191284 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.190553 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.166157 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.198246 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.199720 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.201153 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.201447 8927 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.201764 7963 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:11.202041 8927 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.202595 8913 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.222992 8112 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.204913 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.233002 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.246539 8916 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.246955 7967 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:11.233491 8899 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.208040 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.232115 8927 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.208498 8919 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.250447 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.247846 8910 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.271476 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.271781 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.202548 8905 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.254005 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.247200 8916 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.272588 8905 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.273378 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.274065 8111 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.208652 8911 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.275005 8911 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.275383 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.277406 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.277797 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.278035 8115 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.222806 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.255959 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.264910 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.181854 8897 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:11.297803 8911 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.265379 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.204767 8913 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.258066 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.265455 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.292740 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.292862 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.293322 8910 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.293933 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.294644 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.221925 8909 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.298429 8905 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.208169 8777 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.300204 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.259692 8899 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.301237 7967 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:11.307672 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.308647 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.325593 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.327132 8923 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.328374 8116 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.333315 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.316866 8903 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.353964 8923 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.354462 8903 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.355305 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.335122 8117 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.356408 8903 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.356971 8903 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.357092 8923 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.334244 8928 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.334151 8888 tablet_copy_client.cc:538] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:11.333638 8777 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.327680 8911 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.334748 8909 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.381748 8911 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.383427 8909 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.333482 8055 tablet_copy_service.cc:140] P de4c5ce3ea8345ae9068f825b8d60e19: Received BeginTabletCopySession request for tablet cf8a24d3c225486fa9270926ffe6a40e from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:46718)
I20260616 19:58:11.384176 8055 tablet_copy_service.cc:161] P de4c5ce3ea8345ae9068f825b8d60e19: Beginning new tablet copy session on tablet cf8a24d3c225486fa9270926ffe6a40e from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:46718: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-cf8a24d3c225486fa9270926ffe6a40e
I20260616 19:58:11.339064 8910 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.362258 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.355965 8111 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:11.364538 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.352366 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.365335 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.336480 8913 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.366060 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.368633 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.370854 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.405076 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.405143 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.405732 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.406287 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.406400 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.406719 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.407359 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.407574 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.407748 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.353709 8897 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.421317 8916 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:11.376888 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.376994 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.422709 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.378093 8919 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.378744 8928 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.392499 8899 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.428929 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.378494 8903 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.394279 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.431337 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.432576 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.395485 8777 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.395946 8910 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.396306 8911 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.408731 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.408820 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.372237 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.411619 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.429267 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.433182 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.418308 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.419027 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.422642 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.377604 7934 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.423745 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:11.453826 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 76408.4us
I20260616 19:58:11.456043 8908 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.456595 8908 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.457013 8908 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.432790 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.455662 7934 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.459200 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.463189 8920 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:11.463616 8920 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.463997 8920 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:11.472358 8911 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.472960 8911 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:11.475172 8911 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.476953 8908 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.479033 8908 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.460764 8905 raft_consensus.cc:1064] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:11.462073 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.482697 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.462656 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.484030 8930 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.485965 8905 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:11.487540 7934 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.488389 8930 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.488870 8888 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:11.490635 8918 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.491047 8918 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.491138 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.491433 8918 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.492672 7934 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.494733 8055 tablet_copy_source_session.cc:215] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:11.494782 8903 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.495762 8910 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.496340 8911 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.496423 8910 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.496990 8903 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.497184 8911 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.497634 8903 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.498411 8777 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.498947 8777 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.506695 8918 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:11.509204 8917 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.509660 8917 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.509965 7934 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.511302 8929 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.511736 8929 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.512163 8929 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.510071 8917 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.520658 8929 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.522684 7934 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.526589 8908 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.527464 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.527688 8908 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.528381 8908 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.529800 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.531206 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.543206 8920 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.532267 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.533061 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.533885 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.534736 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.534948 8899 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.535627 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.536352 8929 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.535643 7934 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.537693 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.538357 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.540648 8917 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.541316 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.553735 8920 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.555356 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.555603 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.531513 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.541160 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.541373 8910 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.559944 8045 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.552551 8918 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.560449 8920 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.542737 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.560891 8920 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.560977 8919 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.543671 8777 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.561410 8920 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.531453 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.545279 8044 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.546552 8038 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.555382 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.563123 8903 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.531225 8911 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.563815 8041 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.557130 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.556375 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.549722 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.547219 8036 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:11.558537 8042 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.549299 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.559450 8037 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.559787 8917 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.556532 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.548642 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.539548 8883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf8a24d3c225486fa9270926ffe6a40e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260616 19:58:11.569470 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 47006.6us
I20260616 19:58:11.569324 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.571094 7968 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:11.574194 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.575781 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.578281 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.579030 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.583550 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.584452 7934 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.576367 8930 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.582104 8929 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.588884 7934 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.589937 7934 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.589996 8820 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:11.589959 8922 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.590621 8907 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.590710 8922 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.591053 8907 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.591156 8922 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.591442 8907 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.587090 8918 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260616 19:58:11.595269 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:11.597023 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.597066 7934 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.587914 8929 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.598585 8913 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:11.598599 8929 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.600771 8898 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.602294 8898 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.602794 8898 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.605075 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.592200 8918 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.606390 8918 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.606402 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.617766 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.628192 8928 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.604444 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.635257 7934 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.636062 8917 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.635910 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.636590 8917 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.638383 8917 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.638873 8907 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.641295 8907 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.641863 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.645040 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.646746 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.647390 7962 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:11.647261 8922 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.648427 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.651350 8922 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.654788 8898 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.658284 8917 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.636855 7934 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.662532 8898 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.663401 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.664476 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.664789 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.665844 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.679850 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.666181 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.682258 8043 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.667220 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.667567 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.683415 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.668215 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.668373 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:11.684478 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 47666.7us
I20260616 19:58:11.668546 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:11.669288 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.672792 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.675354 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.676448 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.677122 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.677779 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.679195 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.685108 7934 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:11.688881 8036 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.691540 8922 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.691596 8038 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.690357 8044 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.695923 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.694514 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.692130 8922 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:11.702656 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.702958 8922 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.695613 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.706303 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.738574 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.695541 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:11.746270 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:11.695175 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.693151 8907 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260616 19:58:11.709641 8842 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4060 after lost signal to thread 6785
I20260616 19:58:11.758086 7966 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:11.694871 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.695917 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.760856 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.763357 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.764107 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.767899 8917 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.772267 8908 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.742476 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.773551 8929 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.774272 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:11.693096 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.694981 8918 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.792973 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.805145 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.806123 8038 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.808008 8929 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.813081 8883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52470 (request call id 1) took 2915 ms. Trace:
I20260616 19:58:11.813586 8883 rpcz_store.cc:276] 0616 19:58:08.897670 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:08.897826 (+ 156us) service_pool.cc:224] Handling call
0616 19:58:08.912722 (+ 14896us) ts_tablet_manager.cc:934] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
0616 19:58:11.538188 (+2625466us) tablet_copy_client.cc:368] Tablet copy session begun
0616 19:58:11.652802 (+114614us) tablet_metadata.cc:665] Metadata flushed
0616 19:58:11.652912 (+ 110us) tablet_copy_client.cc:497] Wrote new tablet metadata
0616 19:58:11.812222 (+159310us) tablet_copy_client.cc:418] Wrote new consensus metadata
0616 19:58:11.812550 (+ 328us) tablet_replica.cc:206] Creating consensus instance
0616 19:58:11.812892 (+ 342us) ts_tablet_manager.cc:989] Replica registered.
0616 19:58:11.812998 (+ 106us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":256,"tablet-copy.queue_time_us":14008,"thread_start_us":22981,"threads_started":2}
I20260616 19:58:11.815085 8883 tablet_copy_client.cc:806] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:11.815718 8883 tablet_copy_client.cc:670] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:11.817900 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:11.818692 8842 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4740 after lost signal to thread 7902
I20260616 19:58:11.797708 8922 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.825325 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.817281 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:11.826165 8842 debug-util.cc:398] Leaking SignalData structure 0x7b080027fea0 after lost signal to thread 7994
I20260616 19:58:11.826539 8038 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:11.826843 8922 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:11.827473 8842 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6ee0 after lost signal to thread 8069
I20260616 19:58:11.828473 8908 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.760897 8907 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:58:11.841267 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 83565.4us
I20260616 19:58:11.842376 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.842360 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.843196 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.843264 8011 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.844297 8897 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.845460 8918 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.845741 8011 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:11.846647 8897 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.847040 8897 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.847604 8897 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.841769 8907 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.847069 8011 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.849838 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.850274 8037 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:11.851723 8009 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:11.852870 8842 debug-util.cc:398] Leaking SignalData structure 0x7b08002cdc40 after lost signal to thread 8143
W20260616 19:58:11.854408 8842 debug-util.cc:398] Leaking SignalData structure 0x7b080022ab60 after lost signal to thread 8845
I20260616 19:58:11.856742 8920 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.861122 8919 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.861094 8930 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.865079 8009 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.866148 8930 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.870008 8009 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.872416 8919 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.873308 8897 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.874588 8009 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.875811 8897 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.878158 8084 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:11.879243 8916 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.879626 8084 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:11.879668 8916 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.882020 8920 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.886138 8907 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:11.889844 8919 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.891048 8916 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.895179 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.891667 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:11.901270 8084 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:11.904829 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.909986 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.910715 8910 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.911191 8910 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.911567 8910 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.912393 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.913467 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.914789 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.914871 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.915570 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.916668 8011 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.917484 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.919902 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.920243 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.917208 8085 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:11.918766 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.895221 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:11.925365 8903 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.926491 8913 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.927048 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.927875 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.911934 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:11.923278 8085 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:11.926371 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.931255 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.938937 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.921600 8011 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.932454 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.892743 8934 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.940418 8911 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.922401 8927 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.940815 8934 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.941121 8927 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.941267 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.941565 8927 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.936479 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.941286 8934 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.940593 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.943336 8011 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.947870 8911 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.949245 8883 tablet_copy_client.cc:538] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:11.944489 8905 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:11.952490 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 51307.6us
I20260616 19:58:11.953286 8084 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.954479 8084 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.955634 8084 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:11.961272 8907 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:11.963717 8011 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.964787 8011 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:11.965040 8899 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.966431 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.967631 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.967866 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.968645 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.968880 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.970314 8903 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.970564 8883 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:11.970708 8903 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:11.971189 8903 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.973027 8011 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.974093 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.974614 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.977263 8011 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.978729 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.979245 8011 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.980671 8897 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.981210 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.981500 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.980593 8905 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:11.982888 8905 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:11.983345 8905 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:11.983451 8930 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:11.992991 8011 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:11.993652 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:11.997579 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.001041 8916 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.004431 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.006276 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:12.010401 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 62801.8us
I20260616 19:58:12.010823 8011 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.011811 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.012624 8011 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.013674 8085 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:11.985406 8899 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.022476 8011 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.024829 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.025869 8940 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:12.026386 8940 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.031109 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.032255 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.032519 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.033380 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.033767 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.033802 8934 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.034497 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.034969 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.024354 8777 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:12.027161 8930 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.027711 8888 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:12.038033 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.040756 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.039937 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.032680 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:12.038677 8934 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.049309 8934 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.049760 8934 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.050221 8934 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.020589 8919 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.020615 8911 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.050129 8916 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.052310 8916 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.050051 8910 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.041720 8940 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:12.047932 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 92365us
I20260616 19:58:12.050726 8888 tablet_bootstrap.cc:492] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:12.055733 8910 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:12.060570 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.066064 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:12.068363 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 45972.7us
I20260616 19:58:12.070122 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.073091 8011 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.070370 8084 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:12.074383 8011 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.059419 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.076263 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.077703 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.078896 8888 ts_tablet_manager.cc:1403] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.590s user 0.025s sys 0.008s
I20260616 19:58:12.058987 8927 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.081207 8941 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.076318 8911 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.084718 8913 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.085127 8913 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.085469 8913 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.087035 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.089705 8927 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.090922 8115 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.090782 8903 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.092459 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.095273 8934 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.097458 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.098248 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.099495 8112 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.111732 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.111992 8934 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.099272 8888 raft_consensus.cc:359] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:12.101459 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:12.101665 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.115973 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.116568 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.117180 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.118274 8935 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.104770 8905 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.119879 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.120613 8936 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.121740 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.123832 8936 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.123694 7935 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.126019 8935 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.128655 8903 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:12.127102 8905 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.114673 8888 raft_consensus.cc:740] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:12.130952 7935 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.130432 8888 consensus_queue.cc:260] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:12.133069 7935 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.108269 8011 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.132445 8907 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.134136 8941 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.135573 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.138100 8011 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.138319 8084 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:12.140440 8011 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.142082 8084 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:12.142462 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 128854us
I20260616 19:58:12.143692 8084 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:12.138782 8941 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:12.144017 8777 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.123684 8940 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.140377 8907 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.144448 8943 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.149565 8943 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.150092 8943 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.152002 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.150243 8084 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:12.146904 8939 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.146690 8110 raft_consensus.cc:1240] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.148953 8905 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.154238 8939 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.169481 8939 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.167109 8011 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.155500 8911 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.162297 8898 consensus_queue.cc:1059] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260616 19:58:12.153879 8084 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:12.173077 8011 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:12.160825 8936 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260616 19:58:12.178788 8930 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.179183 8936 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.179586 8936 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.179895 8011 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.180722 8777 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.180498 8936 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.168443 7935 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.174604 8942 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.158685 8110 raft_consensus.cc:1240] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.160669 8085 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.163627 8920 consensus_queue.cc:1059] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
W20260616 19:58:12.182281 8908 consensus_queue.cc:1175] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260616 19:58:12.183995 8085 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.185691 8920 raft_consensus.cc:3055] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.186282 8920 raft_consensus.cc:740] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.186472 8936 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.186729 8085 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.154518 8934 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.177772 8898 raft_consensus.cc:3055] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.179207 8930 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:12.184697 8916 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.185688 8927 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.191052 8907 consensus_queue.cc:1059] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:12.191928 8908 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.192932 8927 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.193125 8907 raft_consensus.cc:3055] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.193410 8927 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.193554 8907 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.193923 8927 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.194689 7935 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.194458 8907 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.195616 8907 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.165637 8910 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:12.196310 8918 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.190310 8938 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:12.187873 8934 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.197499 8934 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.181947 8011 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.189575 8085 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.156754 8905 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.200763 8905 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.189028 8920 consensus_queue.cc:260] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.201939 8920 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.191123 8930 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.188736 8898 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.205876 8916 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.207243 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:12.165906 8913 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.208001 8942 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.208525 8942 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:12.208134 8898 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.208899 8942 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.195765 7935 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.209525 8898 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.214545 8085 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.215561 7935 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.206776 8910 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:12.219167 8938 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.222954 8918 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.216939 8908 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.209643 8940 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.226449 7935 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.227555 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.228462 8918 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:12.229013 7935 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.229820 8918 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.236115 7935 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.236961 8918 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.238267 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.239148 8918 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.240413 7935 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.241196 8918 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.242264 7935 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.243032 8918 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.246536 7935 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.248318 8918 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.248453 8913 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.249851 8927 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.250685 7935 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.251466 8918 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.252525 7935 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.253273 8918 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.254451 7935 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.255172 8918 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:12.256181 8908 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.257192 7935 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.259382 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.262326 8934 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.260876 8918 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.263425 8908 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.266202 8934 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.273326 8930 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.252053 8927 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:12.274871 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 121332us
I20260616 19:58:12.275523 7935 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.276307 8888 ts_tablet_manager.cc:1434] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.197s user 0.015s sys 0.002s
I20260616 19:58:12.277192 8939 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.271605 8942 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.297946 8942 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:12.300493 8939 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.302006 8905 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.304494 8905 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:12.293606 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 79095.9us
I20260616 19:58:12.294493 8084 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.295190 7935 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.307308 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.307562 7935 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.289956 8883 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:12.308552 8939 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.309263 8907 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.309906 8907 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.308717 7935 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.310465 8084 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.311337 7935 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.311626 8913 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:12.312119 8936 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.312345 8938 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:12.312789 8938 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:12.312820 8936 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.313328 8938 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.314410 7935 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.312135 8084 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.316839 8009 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.319061 8009 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.320214 8084 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.322387 8009 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.324334 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.325192 8009 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.326581 8084 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.327466 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.326736 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.330588 8009 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.305470 8920 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.283887 8943 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.325054 8913 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.320885 8911 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.339843 8907 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.332296 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.332911 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.343189 8084 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.343525 8916 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.344331 8939 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.328759 8910 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.334301 8903 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.347672 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.349130 8045 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.350620 8905 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:12.334762 8936 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:12.354611 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.337278 8110 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.338503 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.355341 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.338881 7961 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:12.339057 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.339218 7935 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.357026 8043 raft_consensus.cc:1240] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.357344 7935 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.358377 7935 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.358827 8040 raft_consensus.cc:1240] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.359313 7935 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.359581 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.337373 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.360576 8935 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.360833 8918 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.361155 8935 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.361487 8918 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.361529 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:12.362666 7935 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.317251 8934 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.343264 7968 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:12.344930 8939 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.333510 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.333884 8777 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.335059 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.355782 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.363858 7935 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.341944 8942 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.367239 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.371407 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.378042 7935 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.340858 8009 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.378608 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:12.388243 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:12.389024 8903 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:12.390014 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 49230us
I20260616 19:58:12.390812 8820 raft_consensus.cc:1217] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260616 19:58:12.391175 8009 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.391882 8903 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.393038 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.386130 8918 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.394109 8910 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.395327 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.395335 8009 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.397198 8942 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.366578 8908 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.398269 8908 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.398715 8113 raft_consensus.cc:1240] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.399974 8930 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:12.381603 8943 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.381953 8920 consensus_queue.cc:1059] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260616 19:58:12.384887 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.385677 7969 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:12.387667 7935 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.380631 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.402501 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.402388 8936 consensus_queue.cc:1059] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:12.403475 8936 raft_consensus.cc:3055] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.403841 8936 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.404682 8936 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.385807 8883 tablet_bootstrap.cc:492] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:12.405817 8922 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.404548 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.406544 8883 ts_tablet_manager.cc:1403] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.436s user 0.027s sys 0.018s
I20260616 19:58:12.404101 8907 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.407449 8119 raft_consensus.cc:1240] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.406896 8908 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.407642 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.408875 8907 consensus_queue.cc:1059] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.408764 8908 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.410054 8908 raft_consensus.cc:3055] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.410781 8908 raft_consensus.cc:740] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.405982 8936 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.411716 8908 consensus_queue.cc:260] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.406270 8922 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.413757 7935 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.417533 8922 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.412104 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.409329 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.420069 8935 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.423924 8922 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.425189 8129 tablet_copy_service.cc:342] P 6827184e3cef42c2ada15b2d31afee4b: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-2bf8afc79e3c417c814ac2fb57557096 received from {username='slave'} at 127.0.0.1:34110
I20260616 19:58:12.425690 8129 tablet_copy_service.cc:434] P 6827184e3cef42c2ada15b2d31afee4b: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-2bf8afc79e3c417c814ac2fb57557096 on tablet 2bf8afc79e3c417c814ac2fb57557096 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:12.428326 8946 consensus_queue.cc:1059] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.429932 7966 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:12.430225 8938 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.432298 8938 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:12.435135 7935 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.435928 8936 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.435992 8940 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.436488 8940 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.437273 8940 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.438400 8940 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.439865 8927 consensus_queue.cc:1059] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.441571 8934 raft_consensus.cc:3055] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.441977 8905 raft_consensus.cc:1064] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:12.442193 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.443781 7965 raft_consensus.cc:1240] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.444653 7969 raft_consensus.cc:1240] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.445489 7963 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.446177 7969 raft_consensus.cc:1240] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.441913 8934 raft_consensus.cc:740] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.456128 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.413223 8908 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.458808 8934 consensus_queue.cc:260] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.463330 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:12.465974 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:12.471818 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.473615 7966 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.475567 8883 raft_consensus.cc:359] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:12.476424 8883 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:12.477037 8883 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:12.478020 7965 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.481559 7965 raft_consensus.cc:1240] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.483537 7965 raft_consensus.cc:1240] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.485033 7963 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.485298 8009 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.491704 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.486928 8910 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.500762 7963 raft_consensus.cc:1240] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.499881 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.479295 8883 ts_tablet_manager.cc:1434] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.065s user 0.003s sys 0.000s
I20260616 19:58:12.509608 7970 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.529017 7962 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.529618 7966 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:12.530285 7963 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.530799 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.531472 7970 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.532214 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.532543 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:12.533319 8910 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.534659 8910 consensus_queue.cc:1059] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.535878 8930 raft_consensus.cc:3055] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.536226 8930 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:58:12.537999 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 108980us
I20260616 19:58:12.541788 8115 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.537340 8930 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.543743 8930 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.545527 8911 consensus_queue.cc:1059] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.546862 8011 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.547516 8911 raft_consensus.cc:3055] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.547871 8911 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.548604 8911 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.549525 8119 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:12.549819 8911 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.551023 8905 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:12.552278 8935 consensus_queue.cc:1059] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.552554 8942 consensus_queue.cc:1059] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.553337 8935 raft_consensus.cc:3055] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.553534 8942 raft_consensus.cc:3055] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.553716 8935 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.553965 8942 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.554778 8115 raft_consensus.cc:1240] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.554677 8935 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.554225 8119 raft_consensus.cc:1240] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.555891 8935 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.554183 8910 consensus_queue.cc:1059] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260616 19:58:12.559419 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 129814us
I20260616 19:58:12.562291 8899 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.563017 8898 consensus_queue.cc:1059] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:12.562337 8941 consensus_queue.cc:1059] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260616 19:58:12.554791 8942 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.567768 8899 consensus_queue.cc:1059] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.568027 8903 consensus_queue.cc:1059] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260616 19:58:12.570959 8942 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.578476 8055 tablet_copy_service.cc:342] P de4c5ce3ea8345ae9068f825b8d60e19: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-cf8a24d3c225486fa9270926ffe6a40e received from {username='slave'} at 127.0.0.1:46718
I20260616 19:58:12.578989 8055 tablet_copy_service.cc:434] P de4c5ce3ea8345ae9068f825b8d60e19: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-cf8a24d3c225486fa9270926ffe6a40e on tablet cf8a24d3c225486fa9270926ffe6a40e with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:12.579753 8777 raft_consensus.cc:3055] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.580298 8777 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.583189 8934 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.584713 8777 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.586017 8777 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.593201 8085 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:12.583829 8911 raft_consensus.cc:3055] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.596971 8911 raft_consensus.cc:740] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.597841 8911 consensus_queue.cc:260] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.599103 8911 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.594851 8927 consensus_queue.cc:1059] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:12.583410 8903 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.600448 8903 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.601353 8903 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.602108 8946 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:12.603467 8119 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.603410 8946 raft_consensus.cc:697] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.602154 8927 raft_consensus.cc:3055] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.604826 8927 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.604727 8946 consensus_queue.cc:237] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.605861 8927 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.606185 8939 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:12.608695 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:12.613399 8085 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.617400 8934 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.617705 8922 raft_consensus.cc:3055] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:12.618184 8922 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.618223 8114 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:12.618680 8085 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.620764 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.610517 8927 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.621191 8940 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:12.619115 8922 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.639075 8037 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:12.641070 8922 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:12.610585 8903 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.625982 8943 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.626910 8907 consensus_queue.cc:1059] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.627537 8936 consensus_queue.cc:1059] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.628464 8045 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.627928 8039 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.649202 8940 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.650099 8943 consensus_queue.cc:1059] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.651124 8934 raft_consensus.cc:3055] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.651804 8934 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.651925 8943 consensus_queue.cc:1059] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.650408 8940 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.653141 8938 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.653505 8938 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.653542 8934 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.654429 8938 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.655299 8934 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.655416 8938 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.629863 8935 consensus_queue.cc:1059] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.632870 8085 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.657470 8085 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.658923 8085 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.581490 8910 consensus_queue.cc:1059] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260616 19:58:12.663360 8918 raft_consensus.cc:3055] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.660327 8949 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.664518 8936 raft_consensus.cc:3055] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.664870 8936 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.665992 8085 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:12.666778 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 100232us
I20260616 19:58:12.670233 8903 consensus_queue.cc:1059] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.670843 8930 consensus_queue.cc:1059] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.671273 8777 raft_consensus.cc:3055] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.671830 8777 raft_consensus.cc:740] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.671723 8942 consensus_queue.cc:1059] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:12.672053 8930 raft_consensus.cc:3055] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.672690 8942 raft_consensus.cc:3055] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:12.672750 8930 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.673137 8942 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.672847 8777 consensus_queue.cc:260] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.671274 8903 consensus_queue.cc:1059] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260616 19:58:12.674270 8777 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.669752 8911 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260616 19:58:12.674146 8942 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.663765 8918 raft_consensus.cc:740] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.661375 8950 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:12.678099 8085 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.664882 8948 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.688030 8942 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:12.691781 8085 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.674165 8903 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.698455 8777 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.689704 8916 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.704320 8908 raft_consensus.cc:3055] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.699921 8950 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:12.699208 8903 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.700125 8899 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.711122 8908 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.713616 8114 raft_consensus.cc:1240] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:12.677134 8936 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.714588 8903 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.719944 8903 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.729547 8942 raft_consensus.cc:3055] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.721068 8908 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.715283 8899 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.716054 8950 raft_consensus.cc:697] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.730047 8942 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.730943 8908 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.732198 8899 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.732517 8950 consensus_queue.cc:237] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.733350 8899 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.728533 8936 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.716653 8930 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.702240 8940 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.737443 8930 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.699708 8955 consensus_queue.cc:1059] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260616 19:58:12.718938 8918 consensus_queue.cc:260] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.745324 8955 raft_consensus.cc:3055] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.753993 8820 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:12.745783 8955 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.762018 8819 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:12.772723 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:12.758028 8947 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260616 19:58:12.781154 8941 raft_consensus.cc:3055] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.781726 8941 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.782709 8941 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.760495 8918 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.794657 8922 consensus_queue.cc:1059] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:12.801690 8898 raft_consensus.cc:3055] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.802109 8898 raft_consensus.cc:740] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:12.802443 8039 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:12.803429 8898 consensus_queue.cc:260] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.768874 8955 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:12.807657 8898 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.808557 8948 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:12.797963 8941 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
W20260616 19:58:12.821039 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 156039us
I20260616 19:58:12.759591 8942 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:12.824338 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 130921us
I20260616 19:58:12.836745 8085 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.842060 8085 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.843281 8085 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:12.835780 8818 raft_consensus.cc:1217] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260616 19:58:12.846390 8934 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.854704 8009 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:12.855692 8899 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.859907 8009 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:12.846608 8940 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:12.842041 8943 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
W20260616 19:58:12.861785 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 186042us
I20260616 19:58:12.865398 7963 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:12.863278 8943 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:12.888067 8943 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.878599 8954 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:12.885371 8934 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:12.886945 8955 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.956687 8954 raft_consensus.cc:697] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.972275 8777 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:12.973114 8777 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:12.891281 8942 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.976562 8777 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:12.992571 8927 raft_consensus.cc:1064] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:12.993412 8777 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:12.995101 8927 consensus_queue.cc:237] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:13.012699 7867 catalog_manager.cc:5696] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:13.021508 7867 catalog_manager.cc:5696] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:13.034322 7867 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:12.991297 7963 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:12.988687 8954 consensus_queue.cc:237] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:13.076716 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 53402.4us
I20260616 19:58:13.113552 8039 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.121469 8039 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.128379 8946 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.134289 8819 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:13.134369 7969 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.136916 8963 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:13.137802 7963 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:13.139266 7965 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.144346 8913 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:13.149570 8947 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:13.151528 8041 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.151588 8037 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260616 19:58:13.165733 8961 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:13.171531 8820 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.171538 8913 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260616 19:58:13.181209 8961 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.183051 8819 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.201723 8041 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.211259 7969 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.237449 8041 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.241390 7963 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.250519 8041 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.252528 8972 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:13.258217 8913 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260616 19:58:13.298448 7866 catalog_manager.cc:5696] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.303737 7866 catalog_manager.cc:5696] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 2 to 3, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.323542 8039 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.347334 8115 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:13.367178 8039 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.370359 8819 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260616 19:58:13.369623 7966 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.373085 8927 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:13.376489 8042 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260616 19:58:13.378500 8927 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:13.382571 8946 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:13.382529 7866 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:13.385922 7866 catalog_manager.cc:5696] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:13.380389 8973 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260616 19:58:13.397536 8913 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.414240 8039 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.428937 8819 raft_consensus.cc:2955] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.434734 7963 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.446915 8955 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:13.444664 8961 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260616 19:58:13.452358 7963 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.449817 8955 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.459125 8955 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.459785 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.460544 7965 raft_consensus.cc:2393] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:13.460916 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.463171 8009 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:13.469970 8962 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.477519 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:13.478201 8115 raft_consensus.cc:1275] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.481391 8955 consensus_queue.cc:1048] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:13.483057 8119 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:13.480063 8114 consensus_queue.cc:237] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:13.495721 8820 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260616 19:58:13.496178 8962 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:13.497850 7963 raft_consensus.cc:1275] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.497913 7969 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260616 19:58:13.502790 8927 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:13.508529 8972 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:13.512094 8041 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:13.504702 8962 consensus_queue.cc:1048] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260616 19:58:13.520368 7867 catalog_manager.cc:5696] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 3 to 4, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.526327 8939 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:13.526705 8913 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.537349 7963 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260616 19:58:13.535946 8819 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260616 19:58:13.557974 8916 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:13.570750 7969 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.568176 8820 raft_consensus.cc:2955] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.594323 7953 tablet_service.cc:1558] Processing DeleteTablet for tablet fd3a274b9cd54eea9bdf95a6e2c9e13a with delete_type TABLET_DATA_TOMBSTONED (TS b022cac8cb5e46b7a39a748e81e257a7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49182
I20260616 19:58:13.595938 7865 catalog_manager.cc:5696] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:13.602591 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 5f64cb36408849c989ecda8d19ec4269 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:13.581211 8946 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260616 19:58:13.608814 8927 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.620095 7969 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.628715 8819 raft_consensus.cc:2955] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.655650 9000 tablet_replica.cc:333] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: stopping tablet replica
I20260616 19:58:13.656993 9000 raft_consensus.cc:2243] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:13.657825 9000 raft_consensus.cc:2272] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:13.663079 7868 catalog_manager.cc:5696] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.665875 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 2bf8afc79e3c417c814ac2fb57557096 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:13.667232 9000 ts_tablet_manager.cc:1916] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:13.676963 8028 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f64cb36408849c989ecda8d19ec4269 with delete_type TABLET_DATA_TOMBSTONED (TS de4c5ce3ea8345ae9068f825b8d60e19 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:13.690115 9000 ts_tablet_manager.cc:1929] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260616 19:58:13.690718 9000 log.cc:1199] T fd3a274b9cd54eea9bdf95a6e2c9e13a P b022cac8cb5e46b7a39a748e81e257a7: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root/wals/fd3a274b9cd54eea9bdf95a6e2c9e13a
I20260616 19:58:13.693571 7855 catalog_manager.cc:5027] TS b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): tablet fd3a274b9cd54eea9bdf95a6e2c9e13a (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:13.696269 7868 catalog_manager.cc:5696] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 3 to 4, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New cstate: current_term: 1 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:13.696234 9001 tablet_replica.cc:333] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: stopping tablet replica
I20260616 19:58:13.702467 9001 raft_consensus.cc:2243] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:13.707537 9001 raft_consensus.cc:2272] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:13.715706 8028 tablet_service.cc:1558] Processing DeleteTablet for tablet 2bf8afc79e3c417c814ac2fb57557096 with delete_type TABLET_DATA_TOMBSTONED (TS de4c5ce3ea8345ae9068f825b8d60e19 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:13.716536 9001 ts_tablet_manager.cc:1916] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:13.724637 8907 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:13.725186 8907 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.727162 8907 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.728511 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.729174 8114 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:13.730403 7934 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:13.730914 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.731545 8037 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:13.733603 8907 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:13.734078 8907 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:13.734460 8962 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:13.734470 8907 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.735119 8962 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.737386 8962 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.739518 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.740490 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.741065 7969 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:13.743901 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:13.746363 8962 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:13.751377 8955 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:13.754608 8955 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.756580 8955 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.758649 8962 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:13.760352 8962 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.756646 9001 ts_tablet_manager.cc:1929] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260616 19:58:13.759948 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.759284 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.771229 9001 log.cc:1199] T 5f64cb36408849c989ecda8d19ec4269 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root/wals/5f64cb36408849c989ecda8d19ec4269
I20260616 19:58:13.771898 7969 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:13.773311 9001 tablet_replica.cc:333] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: stopping tablet replica
I20260616 19:58:13.773386 7854 catalog_manager.cc:5027] TS de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): tablet 5f64cb36408849c989ecda8d19ec4269 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:13.774367 8011 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:13.774468 9001 raft_consensus.cc:2243] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:13.775472 8955 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:13.775839 8955 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:13.776072 9001 raft_consensus.cc:2272] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:13.776216 8955 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.780565 9001 ts_tablet_manager.cc:1916] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:13.798653 8954 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:13.799324 8898 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:13.799208 8954 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.799901 8898 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.801354 8954 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.802053 8898 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.805357 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.806012 8115 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:13.807073 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.807691 7966 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:13.808651 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.809455 8045 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:13.810465 7934 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:13.810979 8011 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:13.811323 8898 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:13.811756 8954 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:13.812176 8954 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:13.812532 8954 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.811771 8898 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:13.816635 8898 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.822788 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.825919 9001 ts_tablet_manager.cc:1929] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260616 19:58:13.828783 8918 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:13.834239 9001 log.cc:1199] T 2bf8afc79e3c417c814ac2fb57557096 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root/wals/2bf8afc79e3c417c814ac2fb57557096
I20260616 19:58:13.834312 8918 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.835224 8955 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.836226 7854 catalog_manager.cc:5027] TS de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): tablet 2bf8afc79e3c417c814ac2fb57557096 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:13.837052 8918 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.837177 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.837986 8037 raft_consensus.cc:2393] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:13.839594 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.840237 8114 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:13.841252 7934 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:13.842190 8918 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:13.842569 8918 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:13.842931 8918 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.830235 8898 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.854357 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:13.854543 8898 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.855090 8045 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:13.855969 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:13.851823 8918 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.863099 8907 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.863444 8115 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:13.864796 7934 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:13.865501 8898 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:13.866024 8961 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.866569 8961 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.867810 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:13.868631 8045 raft_consensus.cc:2393] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:13.868734 8961 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.870880 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.871253 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.872311 8045 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:13.871706 7966 raft_consensus.cc:2393] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:13.873631 8084 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:13.874583 8961 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:13.875051 8961 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.875501 8961 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:13.878762 8913 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.880692 8918 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.833132 8962 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.879436 8972 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.884516 8972 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.884927 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:13.885655 7966 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.886572 8972 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.881938 8927 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.887410 8927 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.889307 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.889890 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.889369 8927 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.890019 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.879716 8913 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.890951 7965 raft_consensus.cc:2393] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:13.891559 8045 raft_consensus.cc:2368] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:13.892282 8913 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.893119 8927 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.895134 8927 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.893682 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.896257 7965 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:13.898502 8927 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.899276 9004 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:13.899957 8084 leader_election.cc:400] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:13.899781 9004 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.900300 8084 leader_election.cc:403] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:13.898795 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.899137 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.905000 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.907266 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.907961 8041 raft_consensus.cc:2368] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:13.911125 8084 leader_election.cc:400] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:13.911410 8084 leader_election.cc:403] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:13.914678 7965 raft_consensus.cc:2393] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:13.857607 8954 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.918644 8934 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.920645 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.924496 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:13.925215 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:13.925724 7965 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.926419 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:13.928097 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:13.928666 8042 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.916050 8045 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:13.933344 7965 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:13.935835 8961 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.936551 8011 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:13.937517 8934 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:13.938678 9004 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.938920 8907 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.939195 9004 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:13.939249 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:13.939813 9004 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.916229 8037 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:13.940264 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.940881 8116 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:13.940267 8954 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.941891 7934 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:13.918736 8973 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.942674 8973 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.943087 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.943686 9003 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:13.943770 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.944130 9003 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:13.944423 8116 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:13.944572 9003 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.944669 8973 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.945542 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:13.944456 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:13.946141 8118 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.946739 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.947455 7969 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:13.935765 8042 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:13.948240 7934 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:13.950606 8918 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:13.941870 9004 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.949957 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:13.951511 9004 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:13.951897 9004 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:13.952040 7965 raft_consensus.cc:2393] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:13.952332 9004 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:13.914712 8927 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.948892 8119 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:13.956885 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:13.960650 8009 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:13.923936 8955 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.964020 8118 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:13.938261 8961 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.970733 8084 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:13.971308 7934 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:13.957234 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:13.973033 8039 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:13.952993 8918 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.975018 8918 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.975490 8918 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:13.970069 8934 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.976032 9004 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.976632 8934 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.978367 8037 raft_consensus.cc:2393] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:13.978335 9004 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:13.978570 8934 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:13.979399 8041 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:13.980638 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:13.981352 7965 raft_consensus.cc:2393] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:13.981379 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:13.982206 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:13.983258 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:13.983945 8118 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:13.985178 7934 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:13.993101 9006 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:13.993928 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 57459.8us
I20260616 19:58:13.994352 8119 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:13.995844 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:13.996196 7934 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:13.996554 7965 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:13.978974 9004 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:13.953805 9005 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:13.964102 8946 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.938271 8962 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.005090 7965 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:13.990361 8898 raft_consensus.cc:697] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:13.975968 8918 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:13.988514 8916 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.989681 8934 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:13.988641 8961 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:13.997331 8907 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:14.018198 8955 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:14.007782 8954 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:58:14.066049 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 83948.8us
I20260616 19:58:14.065667 8907 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:14.065313 8934 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.065416 9007 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:14.066668 8916 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.069818 8954 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.079111 8918 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.079468 8954 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.073468 9008 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.079705 8918 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:14.075373 8934 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.080430 8918 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:14.081367 8918 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.083143 8898 consensus_queue.cc:237] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.074786 8907 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.069602 9010 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:14.067097 8961 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.017922 9006 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.029495 8946 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.105127 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:14.028636 8973 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.079435 8916 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.134800 8041 raft_consensus.cc:2393] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:14.073928 7966 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:14.074245 9013 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260616 19:58:14.110783 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 114716us
I20260616 19:58:14.069461 8955 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.101078 9007 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:14.085718 9010 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.059298 9011 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.147758 9009 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:14.154779 9009 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.155418 8955 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.156734 9009 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.145797 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.148102 8962 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.161515 9019 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.163887 8927 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:14.165274 9002 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.102102 9008 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.146029 9013 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.167531 9018 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.170184 8115 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.170423 8934 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.148252 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.153437 9003 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.167858 8961 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.141121 8973 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.159565 8955 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.159708 9005 raft_consensus.cc:697] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:14.160046 9009 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:14.196267 9018 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.203967 8907 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.238555 8907 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.239540 9005 consensus_queue.cc:237] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.240350 9010 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.241060 9007 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.185911 8042 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:14.230140 8037 raft_consensus.cc:2393] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:14.204286 9019 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.235356 9003 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.236881 9018 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.195848 9013 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.159615 9011 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.192528 8962 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:14.257125 9019 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.257548 8114 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:14.243551 9009 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.235412 8973 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:14.263419 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 194136us
I20260616 19:58:14.264439 8084 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:14.265868 8084 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:14.286865 9018 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.287528 9018 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.292809 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:14.293395 8041 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.296098 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.305385 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.298087 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.306887 8043 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.307928 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.308625 8043 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.296762 8040 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.312117 9019 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.211640 8955 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.317312 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.317812 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.325235 8036 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.325564 7965 raft_consensus.cc:2393] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:14.255103 9006 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.204674 9002 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.326190 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.303563 8973 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:14.290962 8946 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.315644 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.329986 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.303918 9003 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.340893 8044 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.324187 8913 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:14.343423 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.346658 8036 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.347002 8041 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:14.290642 9011 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.347313 7934 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:14.349810 7970 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:14.350798 7963 raft_consensus.cc:2393] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:14.380177 9025 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.380995 8907 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:14.380837 9025 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.329852 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.331523 8973 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.326292 9019 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.383477 9017 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.344249 9014 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.337234 9004 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:14.383472 9025 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.387466 9011 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.344789 9009 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.378386 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.396661 8973 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.378865 8934 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.393184 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.348560 9003 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.395430 8913 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:14.446786 8898 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.445439 9025 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.428119 8039 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:14.444049 9018 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.447110 9027 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.348121 9006 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:14.481146 8955 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.429760 9014 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.494038 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.474805 9021 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.445180 9017 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.512701 8036 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.513083 9024 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.494917 8038 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.500537 7964 raft_consensus.cc:2393] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:14.469071 8907 raft_consensus.cc:697] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:14.491161 9011 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.527931 8115 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.502388 8962 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.538383 9037 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.538313 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:14.538856 9036 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.539014 9037 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.539444 9036 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.541103 9037 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.542162 9036 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.511487 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.483127 9004 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.539366 8039 raft_consensus.cc:2368] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:14.528466 8946 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.527540 9018 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.529125 9002 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.556191 9018 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.501259 8898 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:14.501325 9025 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.530926 8918 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.511951 9006 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.545748 8927 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.552867 8946 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.558746 9018 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.585013 7868 catalog_manager.cc:5696] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:14.584563 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:14.586673 8119 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.586350 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.588533 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.589083 8112 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:14.589335 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:14.589826 8116 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.594380 8119 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:14.553771 8042 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:14.571244 9019 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.599483 9030 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.599560 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.512549 9027 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.601347 8119 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:14.554881 8913 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.605604 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.606369 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.607148 8117 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.605082 9022 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.578363 9017 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.582835 9010 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.585867 8961 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.598248 8962 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.518049 9021 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.547108 9024 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.609539 8907 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.600654 9018 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.580428 8918 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.585168 8927 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.629921 8116 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:14.626322 8962 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.621100 9006 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.633555 9018 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.618312 8961 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.618538 9010 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.640743 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.644764 8084 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.647513 9024 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.656344 8118 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.616226 9030 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.656845 8084 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:14.660001 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 127573us
I20260616 19:58:14.661255 7935 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.662322 7935 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.658210 9014 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.664400 7935 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.657747 8934 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:14.667107 7935 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.624845 9022 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.669272 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.670073 7935 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.673504 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.674072 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.674213 8112 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.674611 8116 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.675002 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.663950 9036 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260616 19:58:14.679821 9031 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.122s user 0.000s sys 0.001s
I20260616 19:58:14.681252 9045 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:14.681188 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.681721 9045 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.681911 8117 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:14.682102 9045 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.663552 9044 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.661474 9011 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.671940 8898 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.671824 8934 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.670816 9025 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.683579 9036 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:14.684887 9014 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260616 19:58:14.701387 9038 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.083s user 0.000s sys 0.001s
I20260616 19:58:14.697309 9042 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:14.702596 9042 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:14.702970 9042 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.686520 8946 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.688403 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.696539 8934 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.696245 9011 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:14.691291 9043 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:14.699836 9036 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.715797 8084 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.730211 9052 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:14.730594 9052 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.730952 9052 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.703796 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.733130 8084 leader_election.cc:400] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:14.703939 9014 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.735944 8084 leader_election.cc:403] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:14.743988 8084 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:14.744032 9054 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:14.746001 9052 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.727348 9045 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.747241 9054 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:14.747670 9054 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:14.748097 9054 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.748312 9052 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.752504 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.753170 8044 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:14.692306 9044 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.706686 8918 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.718550 9011 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:14.758576 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.759438 8044 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.761899 9022 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:14.762444 8011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s user 0.008s sys 0.031s
W20260616 19:58:14.762763 8011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s user 0.008s sys 0.031s
W20260616 19:58:14.763022 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 743844us
I20260616 19:58:14.757076 8110 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:14.769133 9010 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.769346 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.770023 8044 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:14.771610 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:14.772292 8045 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.772413 8961 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.737108 9043 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:14.755483 9014 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:14.778224 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.778676 9035 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:14.778995 9040 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:14.785032 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:14.785503 9040 raft_consensus.cc:697] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:14.760459 9011 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.782101 9008 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.782439 9036 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:14.784965 9020 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.347s user 0.005s sys 0.003s
W20260616 19:58:14.788489 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 114507us
I20260616 19:58:14.758852 8927 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.765058 9054 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.803799 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.805033 8119 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:58:14.780705 9013 consensus_meta.cc:287] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.525s user 0.005s sys 0.006s
I20260616 19:58:14.790448 8011 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:14.855294 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 64898.4us
I20260616 19:58:14.794963 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.857141 8041 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:14.773758 9018 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.773926 8962 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.760633 9021 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.781544 9030 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.806159 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.806429 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.918143 8116 raft_consensus.cc:2393] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:14.834117 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.836412 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.836725 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.837019 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.835801 7934 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:14.842999 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.783740 9027 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.799844 9035 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:14.779865 9043 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.924645 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.925406 8116 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:14.926503 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.927171 8116 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:14.837323 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:14.916558 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.919188 7961 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
W20260616 19:58:14.928442 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 92692us
I20260616 19:58:14.928727 8119 raft_consensus.cc:2393] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:14.920060 7968 raft_consensus.cc:2393] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:14.919209 7969 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.923307 8112 raft_consensus.cc:2368] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:14.928079 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.932945 8116 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:14.797871 8039 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.791523 9044 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:14.929556 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:14.930940 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.947216 8119 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:14.937476 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.950300 7963 raft_consensus.cc:2368] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:14.853416 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.939366 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.943692 7970 raft_consensus.cc:2368] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:14.953482 8116 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:14.917126 9008 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:14.957744 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.959367 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.960076 8116 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:14.961308 7934 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:14.963541 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.964509 8119 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:14.965850 7934 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:14.945107 8927 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:14.967195 9027 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:14.924260 9035 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:14.971033 9047 log.cc:927] Time spent T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.190s user 0.000s sys 0.000s
I20260616 19:58:14.967599 9027 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:14.927425 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:14.972052 9027 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:14.967597 8927 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.938697 9030 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:14.973841 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.979207 8115 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:14.945562 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:14.917009 9013 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.972528 8042 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:14.985607 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:14.985702 8110 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:14.986721 9036 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:14.985327 8044 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
W20260616 19:58:14.991115 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 321084us
I20260616 19:58:14.973323 9021 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:14.994729 7934 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:14.786427 9040 consensus_queue.cc:237] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:14.976316 9008 raft_consensus.cc:697] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:15.001780 7934 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:15.013574 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.014367 7969 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
W20260616 19:58:15.014582 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 212555us
I20260616 19:58:15.005131 8918 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.005929 9004 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:15.006513 9036 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:15.006659 8045 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.010229 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.010135 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.011308 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.021059 7970 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:15.015295 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.015923 8084 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.016716 8115 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:15.017287 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.017576 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:14.989737 9030 raft_consensus.cc:697] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:15.023416 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.023763 8116 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.003528 9021 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.019690 7966 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.026252 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.026968 8115 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.026523 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.023763 8119 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:14.990543 8973 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.022662 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.031072 8116 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:15.031381 8084 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.040935 8115 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.018319 8918 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.045748 9035 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.027280 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.021565 9036 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.050913 8946 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.024637 9045 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.062760 8111 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.063120 9049 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.063271 9013 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.063526 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.028787 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.053277 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.064510 8113 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:15.065043 8110 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.050652 8918 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.027335 9006 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.025578 8913 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.020195 9050 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:15.028561 8898 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260616 19:58:15.046708 9051 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260616 19:58:15.064647 9056 log.cc:927] Time spent T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Append to log took a long time: real 0.083s user 0.000s sys 0.001s
I20260616 19:58:15.065169 9049 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.027595 9021 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.021970 9004 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.051951 8011 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.061712 8927 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.056517 9012 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:15.070659 9006 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:58:15.160882 9032 log.cc:927] Time spent T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.630s user 0.008s sys 0.005s
I20260616 19:58:15.161262 9004 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.161724 9010 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:15.162142 9010 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:15.162494 9010 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:15.081357 7967 raft_consensus.cc:2393] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.061990 8946 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:15.156757 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 104907us
I20260616 19:58:15.082074 9054 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.073091 8913 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.082126 9051 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:15.065301 9055 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260616 19:58:15.106529 9049 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.073578 9050 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:15.204496 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 178032us
I20260616 19:58:15.078184 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.157092 8898 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.163751 9012 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:15.068758 9008 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.213781 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.189980 8946 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.164695 9006 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.214265 7934 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.216027 8115 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.081640 9030 consensus_queue.cc:237] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.209299 9051 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.217726 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.218303 9036 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.215629 8898 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:15.226613 9040 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.203491 8913 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.227986 9040 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.228420 9040 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.215804 7966 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:15.225804 7934 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.211797 9050 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:15.210995 8973 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.234779 7934 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.237349 8085 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.216132 9012 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:15.189879 9043 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.187156 9042 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.227110 8037 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:15.278246 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:15.287426 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 50152.1us
W20260616 19:58:15.255653 8934 consensus_meta.cc:287] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.539s user 0.010s sys 0.000s
I20260616 19:58:15.307893 8085 leader_election.cc:400] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.308364 8085 leader_election.cc:403] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:15.310086 8085 leader_election.cc:400] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.310437 8085 leader_election.cc:403] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:15.264186 9034 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:15.239830 9036 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:15.265336 8973 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.265501 8898 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:15.270968 9025 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.315630 8946 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.272393 9003 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.295841 8962 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.305979 9004 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.233718 9018 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260616 19:58:15.322824 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.322536 8973 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.339432 8898 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.320940 8918 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:15.336730 9004 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.332837 9036 raft_consensus.cc:697] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:15.306823 8934 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.342645 9061 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.331573 9035 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.332238 9054 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
W20260616 19:58:15.343797 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 531688us
I20260616 19:58:15.346269 8973 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.331790 9025 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.334697 9034 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.336715 8962 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.344975 9041 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.353760 7969 raft_consensus.cc:2393] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.356340 8112 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:15.356312 8009 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.349403 9004 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:15.364284 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 129540us
I20260616 19:58:15.364991 7934 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.365691 9018 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:15.366034 9018 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.366364 9018 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:15.367251 7934 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.369073 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.369987 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.348292 8898 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.372241 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.373402 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.372931 9035 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.389878 8009 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.386900 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.400524 8011 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.389796 9035 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.368688 9041 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.388214 9021 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.399869 9043 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.405628 9035 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.420528 7966 raft_consensus.cc:2393] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.398304 9036 consensus_queue.cc:237] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.405743 8962 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.425823 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.429165 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.408810 9042 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.409375 9025 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.433553 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.416067 8918 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.422207 8946 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.440908 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.442332 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.442353 9025 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.443632 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.443514 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.443666 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.444581 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.445518 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.457106 8036 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:15.445878 8044 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:15.446141 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.458164 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.450393 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.458712 8036 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.415145 9064 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.451251 8934 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.434904 8898 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.455726 8041 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.459084 8118 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.459211 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.458474 8040 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:15.463138 8044 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.457139 8043 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.463753 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.463877 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.464628 7962 raft_consensus.cc:2368] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:15.462383 8038 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:15.464777 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.471899 8044 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.472462 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.456522 9042 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.475924 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.454427 9050 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.477325 8044 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:15.480374 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.480975 8040 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.485119 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.465226 8045 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.485811 8044 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.487887 8044 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:15.460260 9025 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.488292 8118 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.474239 8898 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:15.484727 9027 consensus_meta.cc:287] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.512s user 0.008s sys 0.005s
I20260616 19:58:15.480000 9042 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:15.490612 9063 log.cc:927] Time spent T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.120s user 0.001s sys 0.000s
I20260616 19:58:15.462291 9010 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.489989 7934 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.490490 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.486752 8962 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:15.497030 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.489989 9034 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.493165 8898 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.520309 8043 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.493412 7964 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:15.494392 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.494266 9027 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.507134 9006 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.502313 8918 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:15.520381 8118 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:15.502655 8962 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:15.566185 8962 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:15.473425 9064 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.506012 9043 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.574059 9043 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.575811 9051 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.579679 9006 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.581679 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.584393 8041 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.561973 8118 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:15.570119 8036 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.570730 9008 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.573555 9064 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.586323 9008 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.586812 9004 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.588382 9043 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.588841 9008 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.590127 7970 raft_consensus.cc:2393] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.592960 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.593607 7963 raft_consensus.cc:2368] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.594352 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.594581 8973 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.511776 8913 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.595660 8119 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:58:15.596261 9011 consensus_meta.cc:287] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.794s user 0.003s sys 0.008s
I20260616 19:58:15.596623 9011 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.597208 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.597520 8913 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.597853 7969 raft_consensus.cc:2368] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.598574 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.598696 9011 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.599472 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 539) took 1010 ms. Trace:
I20260616 19:58:15.599656 7965 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.599877 8111 rpcz_store.cc:276] 0616 19:58:14.589460 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:14.595466 (+ 6006us) service_pool.cc:224] Handling call
0616 19:58:15.599322 (+1003856us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54000d7d54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:15.599415 (+ 93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1044570496}
I20260616 19:58:15.603368 8085 leader_election.cc:400] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.608433 8085 leader_election.cc:403] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:15.609889 8085 leader_election.cc:400] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.615097 8085 leader_election.cc:403] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:15.615296 7961 raft_consensus.cc:2368] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:15.611532 9025 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.612058 9042 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.624867 9050 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.604065 9021 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.630971 7868 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 118) took 1316 ms. Trace:
I20260616 19:58:15.638865 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.650759 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.632927 9041 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.643541 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.630951 8039 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:15.617266 9022 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.611030 9066 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:15.616355 8913 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.636835 8962 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.659844 8117 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.668278 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.679617 7968 raft_consensus.cc:2393] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.679683 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.602835 9067 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.680707 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.683863 9010 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.648808 7868 rpcz_store.cc:276] 0616 19:58:14.330363 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:14.339143 (+ 8780us) service_pool.cc:224] Handling call
0616 19:58:15.630914 (+1291771us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:14.685468 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:14.774589 (+ 89121us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:14.774633 (+ 44us) write_op.cc:435] Acquired schema lock
0616 19:58:14.774654 (+ 21us) tablet.cc:662] Decoding operations
0616 19:58:14.774982 (+ 328us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:14.775062 (+ 80us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:14.775098 (+ 36us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:14.775286 (+ 188us) tablet.cc:701] Row locks acquired
0616 19:58:14.775303 (+ 17us) write_op.cc:260] PREPARE: finished
0616 19:58:14.775440 (+ 137us) write_op.cc:270] Start()
0616 19:58:14.775551 (+ 111us) write_op.cc:276] Timestamp: P: 1781639894775398 usec, L: 0
0616 19:58:14.775573 (+ 22us) op_driver.cc:348] REPLICATION: starting
0616 19:58:14.778198 (+ 2625us) log.cc:844] Serialized 540 byte log entry
0616 19:58:15.349223 (+571025us) op_driver.cc:464] REPLICATION: finished
0616 19:58:15.386924 (+ 37701us) write_op.cc:301] APPLY: starting
0616 19:58:15.390553 (+ 3629us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:15.393187 (+ 2634us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:15.405085 (+ 11898us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:15.412545 (+ 7460us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:15.412727 (+ 182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:15.412783 (+ 56us) write_op.cc:312] APPLY: finished
0616 19:58:15.412985 (+ 202us) log.cc:844] Serialized 22 byte log entry
0616 19:58:15.413151 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:15.415421 (+ 2270us) write_op.cc:454] Released schema lock
0616 19:58:15.422410 (+ 6989us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:15.455989 (+ 33579us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:15.461463 (+ 5474us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":28244,"apply.run_cpu_time_us":2025,"apply.run_wall_time_us":58093,"num_ops":1,"prepare.queue_time_us":73642,"prepare.run_cpu_time_us":49438,"prepare.run_wall_time_us":269919,"raft.queue_time_us":90505,"raft.run_cpu_time_us":1869,"raft.run_wall_time_us":52772,"replication_time_us":564155,"thread_start_us":285015,"threads_started":4,"wal-append.queue_time_us":203033}]]}
I20260616 19:58:15.685400 9027 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.685930 9018 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.686517 8962 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.688010 7967 raft_consensus.cc:2368] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.693795 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.694653 7967 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.695500 8085 leader_election.cc:400] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.695896 8085 leader_election.cc:403] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:15.688550 7968 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:15.625766 9068 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.686457 8898 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.623327 9012 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.703481 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.702415 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.705922 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.706681 8117 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.691246 9066 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.710489 7965 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:15.652673 9072 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.714879 7935 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.700325 9033 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.719341 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.722195 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:15.722661 9066 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.722911 7968 raft_consensus.cc:2393] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:58:15.714656 9049 consensus_meta.cc:287] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.502s user 0.007s sys 0.006s
I20260616 19:58:15.725275 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:15.726372 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 336545us
I20260616 19:58:15.716717 9004 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.722980 9012 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.718298 8898 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.701706 9067 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.756083 8009 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.758971 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.759608 8036 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.760736 8009 leader_election.cc:400] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:15.761039 8009 leader_election.cc:403] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:15.762048 8009 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.762931 8009 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.763772 8118 raft_consensus.cc:1275] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:15.766034 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.766223 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.766670 8040 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.723277 8038 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.769323 8036 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.770869 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.771699 8118 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:58:15.760811 9040 consensus_meta.cc:287] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.532s user 0.016s sys 0.003s
I20260616 19:58:15.788003 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.758141 8898 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.759122 8962 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:15.762343 9010 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.765920 9064 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.772316 8918 raft_consensus.cc:697] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:58:15.834216 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 71342.8us
I20260616 19:58:15.827486 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:15.839810 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 439316us
I20260616 19:58:15.794137 8973 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:15.765386 9008 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:15.842548 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:15.788471 9072 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:15.841372 9014 consensus_meta.cc:287] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 1.037s user 0.013s sys 0.000s
I20260616 19:58:15.845041 9006 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:15.827569 9040 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:15.846606 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 60473.9us
I20260616 19:58:15.858381 7934 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:15.847522 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.828706 8898 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.844377 8009 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.834945 9010 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.751508 9049 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.840927 9064 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.856621 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.880923 7963 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.849510 9076 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.882905 9076 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.883277 9076 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.884559 9018 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.885905 9035 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:15.887151 9061 consensus_meta.cc:287] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.531s user 0.004s sys 0.007s
I20260616 19:58:15.872717 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:15.887667 9061 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:15.849056 8037 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:15.888656 7969 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.836791 8962 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.879703 8044 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.887818 8111 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.895660 8044 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.872361 8041 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:15.897230 8118 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.897897 8040 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.852828 9072 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.936689 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.937351 8040 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.850610 9080 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.887943 9035 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.872972 9036 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.898559 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.888521 9064 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.848065 9021 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260616 19:58:15.860634 9006 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.851078 9019 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.872839 9066 consensus_queue.cc:1048] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260616 19:58:15.873061 9010 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:15.897483 8110 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:15.938367 7966 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:15.852265 9077 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.845877 9075 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260616 19:58:15.944655 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.848284 9067 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.949810 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.950376 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:15.899464 9081 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:15.944904 9018 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:15.854950 9014 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.948215 9036 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.946008 9008 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:15.954872 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 74652.2us
I20260616 19:58:15.993485 8009 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:15.992267 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 760) took 1157 ms. Trace:
I20260616 19:58:15.991827 9081 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.995042 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.947364 9027 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.948916 9035 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:15.945837 9080 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.951936 8918 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:16.034010 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.038347 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.038929 8041 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.039034 8114 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:15.957507 9049 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:15.976243 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:15.993672 9036 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.070281 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.088927 8115 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.074373 8112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 507) took 1231 ms. Trace:
I20260616 19:58:16.070562 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 509) took 1062 ms. Trace:
I20260616 19:58:16.090457 7961 rpcz_store.cc:276] 0616 19:58:15.007990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:15.008122 (+ 132us) service_pool.cc:224] Handling call
0616 19:58:16.070315 (+1062193us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b540027bad4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:16.070494 (+ 179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367347968}
I20260616 19:58:16.089882 8112 rpcz_store.cc:276] 0616 19:58:14.807392 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:14.957433 (+150041us) service_pool.cc:224] Handling call
0616 19:58:16.038283 (+1080850us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b5400102a54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:16.038381 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914747264}
I20260616 19:58:15.955947 7864 catalog_manager.cc:5696] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:15.957227 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:15.963810 9061 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:16.078539 9051 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.091993 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.069468 9041 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.004709 9081 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.044242 9067 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:16.102912 8041 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.957928 9019 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.956779 8117 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:15.995432 9018 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.037443 9064 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:15.996841 9008 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.112815 8115 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.037412 9027 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.134521 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:16.136181 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 140168us
I20260616 19:58:16.039536 9035 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.089330 9006 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.106037 8042 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.161607 8043 raft_consensus.cc:2393] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:16.135802 9067 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:15.994390 7964 rpcz_store.cc:276] 0616 19:58:14.835295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:14.854422 (+ 19127us) service_pool.cc:224] Handling call
0616 19:58:15.063405 (+208983us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6a3e 000000000036f859 00007fe8742348e0 00007fe8742fd7c4 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:58:15.992046 (+928641us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54003b1354. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:15.992181 (+ 135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987404544}
I20260616 19:58:15.956885 9077 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.172310 7961 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.137948 9040 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.145220 9082 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.169131 9006 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:16.177452 9071 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.415s user 0.000s sys 0.004s
I20260616 19:58:16.178332 9035 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.179472 9080 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.181090 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 518) took 1170 ms. Trace:
I20260616 19:58:16.209335 7969 rpcz_store.cc:276] 0616 19:58:15.010413 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:15.598380 (+587967us) service_pool.cc:224] Handling call
0616 19:58:16.180899 (+582519us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400002dd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:16.181019 (+ 120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665448832}
I20260616 19:58:16.182300 9018 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.152530 9041 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.183203 9084 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.183974 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 514) took 1174 ms. Trace:
I20260616 19:58:16.184098 9014 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:16.184608 7962 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 513) took 1175 ms (client timeout 1553 ms). Trace:
I20260616 19:58:16.185369 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 521) took 1174 ms. Trace:
I20260616 19:58:16.187901 8084 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:16.189288 8114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 543) took 1517 ms (client timeout 1848 ms). Trace:
I20260616 19:58:16.189744 9019 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.191458 7964 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.191658 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.172658 8037 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:16.192348 9085 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.192822 9043 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.193558 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 523) took 1172 ms. Trace:
W20260616 19:58:16.167399 9079 log.cc:927] Time spent T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.227s user 0.001s sys 0.000s
I20260616 19:58:16.208242 9002 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.134008 9061 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.226308 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.226601 8084 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:16.240265 9023 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:16.225467 7963 rpcz_store.cc:276] 0616 19:58:15.009455 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:15.594141 (+584686us) service_pool.cc:224] Handling call
0616 19:58:16.183795 (+589654us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b540016ba54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:16.183918 (+ 123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654198528}
I20260616 19:58:16.240494 9086 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:16.197842 9082 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.239676 7967 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:16.240752 8040 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.175750 8962 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.242383 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.195516 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.181116 9008 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.244781 7961 rpcz_store.cc:276] 0616 19:58:15.021132 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:16.103296 (+1082164us) service_pool.cc:224] Handling call
0616 19:58:16.103674 (+ 378us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:16.176011 (+ 72337us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:16.193346 (+ 17335us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b540025a154. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:16.193505 (+ 159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153663744}
I20260616 19:58:16.193934 9076 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:16.209204 9022 consensus_meta.cc:287] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.542s user 0.010s sys 0.000s
I20260616 19:58:16.252038 8084 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:16.227633 8114 rpcz_store.cc:269] 0616 19:58:14.671941 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:14.672606 (+ 665us) service_pool.cc:224] Handling call
0616 19:58:16.189122 (+1516516us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001536d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:16.189229 (+ 107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043218176}
I20260616 19:58:16.253819 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.268957 7963 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:16.225375 7970 rpcz_store.cc:276] 0616 19:58:15.011152 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:15.053494 (+ 42342us) service_pool.cc:224] Handling call
0616 19:58:15.581540 (+528046us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:58:16.185196 (+603656us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b54003b0e54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:16.185315 (+ 119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20814592}
I20260616 19:58:16.175704 9067 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.246837 8011 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:16.278774 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:16.207875 9006 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.264767 8043 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.265828 9089 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.266350 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:16.257365 7935 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:16.270018 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.283640 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.270323 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.252856 9014 raft_consensus.cc:2764] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260616 19:58:16.273244 8913 consensus_meta.cc:287] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.594s user 0.007s sys 0.001s
I20260616 19:58:16.266904 9040 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.274529 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.274904 9068 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:16.270228 9080 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260616 19:58:16.275626 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.276067 9022 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.276651 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.311555 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.277014 9076 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.312105 8040 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.254773 9023 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:16.313169 8042 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.313342 9023 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.280200 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:16.282303 9019 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:16.282666 7969 raft_consensus.cc:2368] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:16.284111 8044 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.316073 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.285527 7935 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:16.253177 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 524) took 1030 ms. Trace:
I20260616 19:58:16.276890 8111 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.289265 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.291112 9006 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.256872 9043 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.322867 9043 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.319384 7964 rpcz_store.cc:276] 0616 19:58:15.222789 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:16.190715 (+967926us) service_pool.cc:224] Handling call
0616 19:58:16.191077 (+ 362us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:16.241690 (+ 50613us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:16.251292 (+ 9602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:16.293501 8913 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.286044 9089 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.286166 8973 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:16.295200 8041 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:16.287482 9003 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:16.262729 9088 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.297333 9061 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:16.298269 9040 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.280897 9012 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:16.299440 8039 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:16.301142 9082 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.241007 9084 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.264389 9086 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.254436 9002 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:16.230010 7962 rpcz_store.cc:269] 0616 19:58:15.009215 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:15.013343 (+ 4128us) service_pool.cc:224] Handling call
0616 19:58:16.184395 (+1171052us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b540004c9d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:16.184546 (+ 151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854953344}
I20260616 19:58:16.331928 9012 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:16.247624 9085 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:16.316794 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:16.317240 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.352488 7961 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:16.317952 7935 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.320945 8038 raft_consensus.cc:2368] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.305119 9068 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:16.328205 9089 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.329842 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.328691 9088 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:16.297333 9036 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.317309 9019 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.375824 9019 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:16.356717 9012 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.320202 9018 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:16.329540 9003 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:16.347476 9077 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.330878 8045 raft_consensus.cc:2368] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.376940 9088 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:16.357540 9041 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.427250 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.354008 8898 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.431344 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.431938 7968 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:16.432824 8898 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.358248 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.433948 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.434406 8039 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.359727 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.329952 9043 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.332499 9027 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.367162 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:16.332294 8973 raft_consensus.cc:697] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:16.367691 9004 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.367931 9086 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.369632 9010 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.428094 8038 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:16.370453 9085 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.370277 9072 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.372457 7965 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.332614 9064 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.374289 8040 raft_consensus.cc:2368] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.348129 9035 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.458523 9004 raft_consensus.cc:697] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:16.458789 9035 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.459424 9004 consensus_queue.cc:237] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.332509 8044 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
W20260616 19:58:16.376145 9033 consensus_meta.cc:287] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.655s user 0.009s sys 0.000s
I20260616 19:58:16.279240 8011 leader_election.cc:400] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:16.471499 8011 leader_election.cc:403] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:16.378448 9003 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:16.472175 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 193035us
I20260616 19:58:16.379693 9040 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.384291 9002 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.479848 9033 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.411801 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.380470 9036 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.439047 8009 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:16.488381 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 49417.8us
I20260616 19:58:16.490248 9086 raft_consensus.cc:2764] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260616 19:58:16.490981 9086 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:16.441759 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:58:16.444074 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 84426.9us
I20260616 19:58:16.470353 8085 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:16.445144 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.445480 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.496522 8116 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.446784 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.498054 8115 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:16.496471 8114 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:16.452464 8973 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.454743 8039 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:16.460281 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.460881 9035 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.456557 8934 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:16.375018 8043 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.453831 9027 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:16.526167 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.527267 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.463063 8040 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:16.527801 8043 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:16.464295 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.537380 8044 raft_consensus.cc:1275] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.464728 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.465016 9010 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.465456 8036 raft_consensus.cc:1240] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:16.447480 9043 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.547139 9043 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.423029 9019 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.549006 9019 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.469501 9084 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.401612 9012 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.550946 9012 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:16.476600 7933 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:58:16.553030 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 76508.1us
I20260616 19:58:16.478549 9072 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.481518 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.393015 7868 catalog_manager.cc:5696] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 4, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:16.554711 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.555383 8110 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.556252 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.481681 9022 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260616 19:58:16.505676 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:16.508425 8114 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:16.509260 8119 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.437925 9081 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.520499 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.532124 8116 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.569375 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.569866 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.570564 8116 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:16.538432 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.527397 7934 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:16.541111 8084 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:16.514986 8011 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.553s (SENT) [suppressed 3 similar messages]
I20260616 19:58:16.521397 8927 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:16.560119 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.562700 8118 raft_consensus.cc:1275] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.566614 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.570379 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.577008 8039 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:16.525818 9064 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:16.526319 9027 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:16.571868 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44549.6us
I20260616 19:58:16.571316 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.535643 8043 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:16.521126 9067 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:16.578485 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 72910us
I20260616 19:58:16.515484 9036 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
W20260616 19:58:16.580237 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 66911.1us
I20260616 19:58:16.557327 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.612550 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.614459 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.616743 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:16.618448 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.619710 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.527760 7966 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:16.544977 8113 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.582749 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.587184 7934 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:16.636387 8039 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:58:16.636299 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 49189.3us
I20260616 19:58:16.585420 9027 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:16.637394 9036 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.637984 9036 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:58:16.637985 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 96925.3us
I20260616 19:58:16.639040 9036 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.636400 9003 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.642752 9003 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.643522 7963 raft_consensus.cc:2368] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.643601 8084 leader_election.cc:400] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.588058 8041 raft_consensus.cc:2368] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.572230 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:16.584863 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:16.593499 9045 raft_consensus.cc:2764] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260616 19:58:16.528848 8934 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.578732 9040 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:16.574272 8927 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:16.656392 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:16.577736 8037 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:16.660761 8934 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:16.662844 9081 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.596621 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:16.663769 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 67269.8us
I20260616 19:58:16.664487 8927 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:16.584192 9041 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:16.667217 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.656476 9039 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:16.668056 8118 raft_consensus.cc:2393] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:16.585376 9023 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.668952 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.669235 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.669992 8085 leader_election.cc:400] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.670143 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.569209 9043 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:16.663888 9018 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.648406 8040 raft_consensus.cc:1275] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.648615 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.644996 8084 leader_election.cc:400] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.674525 8084 leader_election.cc:403] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:16.675071 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:16.648597 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 133609us
I20260616 19:58:16.653772 9067 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.675457 8084 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:16.657877 8115 raft_consensus.cc:2368] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:16.659089 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.670369 8085 leader_election.cc:403] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:16.676101 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.670665 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.671273 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.662879 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.679571 8084 leader_election.cc:400] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.647424 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.680094 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.681047 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.682169 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.682862 8114 raft_consensus.cc:2368] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:16.683553 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.664532 7935 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.672262 8110 raft_consensus.cc:1275] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.688793 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.690012 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:16.690235 7935 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.692452 8009 leader_election.cc:400] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:16.692791 8009 leader_election.cc:403] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:16.692662 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.694437 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.695907 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.697542 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.697742 7935 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:16.698299 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.701205 8111 raft_consensus.cc:2368] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:16.701478 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.702020 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.668440 9022 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:16.673987 9045 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:16.698482 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:16.704419 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:16.710245 8084 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:16.705830 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:16.673354 9040 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:16.697664 9067 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:16.682199 8934 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:16.691991 9018 raft_consensus.cc:697] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:16.711287 7935 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.715667 8084 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:16.668196 9039 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.709528 9002 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:16.739368 8009 leader_election.cc:400] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:16.705925 9003 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260616 19:58:16.760632 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:16.772415 8114 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:16.762313 8045 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:16.704357 8898 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.766815 8819 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 LEARNER]: Advancing to term 3
W20260616 19:58:16.768225 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 57021.6us
I20260616 19:58:16.708257 9023 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:16.768572 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:16.687813 8913 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:16.699087 9035 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:16.699306 9088 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
W20260616 19:58:16.761631 9096 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.075s user 0.000s sys 0.000s
I20260616 19:58:16.706028 9041 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:16.703473 8111 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260616 19:58:16.792217 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 76639.3us
I20260616 19:58:16.737072 9045 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.768771 8009 leader_election.cc:403] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260616 19:58:16.816483 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 77304.8us
I20260616 19:58:16.772125 9012 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:16.738610 9061 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.736820 9004 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:16.796984 9090 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.760124 9002 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.810245 9035 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.707952 9080 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:16.833216 9018 consensus_queue.cc:237] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.707546 9098 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.715410 9022 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.809916 8934 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.825564 8898 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:16.827665 7935 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.791774 9075 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.823547 9067 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:16.832279 9040 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.833050 9002 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:16.825098 9012 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.839018 8119 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:16.813330 9041 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.827003 9039 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:16.872416 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 44874.2us
I20260616 19:58:16.840339 9035 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:16.875787 7935 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:16.871248 8934 raft_consensus.cc:697] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:16.903555 8934 consensus_queue.cc:237] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.875075 9041 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:16.890923 9049 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.907743 9049 raft_consensus.cc:697] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:16.908636 9049 consensus_queue.cc:237] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.891541 7935 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:58:16.911821 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 274728us
I20260616 19:58:16.912489 7935 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:16.913187 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:16.913482 8085 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:16.914006 7935 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:16.915587 7935 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:16.916347 8009 leader_election.cc:400] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:16.916754 8009 leader_election.cc:403] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260616 19:58:16.877082 9003 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:16.867865 9088 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260616 19:58:16.876649 9075 raft_consensus.cc:697] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:16.924691 8113 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:16.869185 8927 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.838526 9090 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:16.929801 9030 consensus_queue.cc:1048] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:16.923534 9002 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:16.922585 8913 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260616 19:58:16.938886 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 122756us
I20260616 19:58:16.871577 9068 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:16.923633 9004 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.923031 9043 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:16.920696 9003 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.923183 9067 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:16.961823 8117 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:16.922643 9080 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:16.955394 8913 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:16.955144 9002 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:16.944083 9081 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:17.007567 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.015674 7963 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:16.939392 9027 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.017385 8085 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:16.995368 9003 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:16.944365 9064 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260616 19:58:16.966363 9105 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:16.965864 8898 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.012441 8913 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:17.014537 9002 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:17.031798 7970 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:58:17.034922 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.039017 9068 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.018848 9061 raft_consensus.cc:697] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:17.054370 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 138825us
I20260616 19:58:17.024250 9004 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.027156 9043 raft_consensus.cc:697] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:58:17.072198 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 156968us
I20260616 19:58:17.070011 7963 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:17.020902 9081 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.034741 9097 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:17.078087 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:17.083209 7934 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.029345 9067 consensus_queue.cc:1059] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260616 19:58:17.086349 7934 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:17.081457 8085 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:17.045341 9045 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.093448 8011 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.094077 7934 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.119556 8011 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:17.020556 9075 consensus_queue.cc:237] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.120641 7934 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.086417 8819 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:17.131892 8041 raft_consensus.cc:1275] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:17.087283 9097 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:17.088976 9012 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.092320 9105 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:17.092499 9049 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.118733 9045 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:17.131011 9088 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.164461 9086 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:17.177671 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.179086 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.190582 8043 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:17.190948 8039 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:17.191813 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.192449 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.178179 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.193455 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.178994 8040 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:17.180701 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.188812 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.169243 9097 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:17.118927 9018 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260616 19:58:17.118865 9036 consensus_queue.cc:1059] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
I20260616 19:58:17.170840 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.194191 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.194564 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.194854 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.203333 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:17.208489 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 89007.1us
I20260616 19:58:17.174417 8927 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.175285 9064 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:17.219283 8039 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:17.175901 9045 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:17.179572 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.176396 9118 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260616 19:58:17.229401 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 108840us
I20260616 19:58:17.218297 8011 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.193866 9120 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:17.230997 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:17.230188 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.219614 9064 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:17.220463 9027 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.203874 9105 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.228705 8043 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
W20260616 19:58:17.235473 9117 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20260616 19:58:17.235697 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.233086 9090 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.240880 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.221707 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.242000 9022 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.242161 8946 consensus_queue.cc:1059] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:17.243448 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.244244 8011 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.245374 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.166815 9004 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.247511 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.207850 9067 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:17.222616 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.225374 9045 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:17.253356 8039 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
W20260616 19:58:17.228914 9115 log.cc:927] Time spent T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
I20260616 19:58:17.262542 9080 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260616 19:58:17.227427 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.259330 7935 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.307996 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.308668 9129 raft_consensus.cc:3055] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:17.176247 9061 consensus_queue.cc:237] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:17.302964 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 58806us
I20260616 19:58:17.309993 8011 leader_election.cc:400] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:17.311520 8011 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.178848 9043 consensus_queue.cc:237] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.225850 9126 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:17.263696 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.315375 7967 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:58:17.315467 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 359573us
I20260616 19:58:17.265877 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.230662 9118 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:17.291558 8040 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:17.247196 9125 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.256717 9064 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:17.254822 9088 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260616 19:58:17.297289 8043 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:17.299166 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.302086 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:17.303669 9116 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260616 19:58:17.332515 7965 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:17.304894 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.301522 9067 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.202566 9086 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:17.314872 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 59998.5us
I20260616 19:58:17.236536 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:17.272266 9120 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.328177 8084 leader_election.cc:400] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:17.317947 8973 raft_consensus.cc:3055] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:17.315626 9129 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.318305 9081 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.315155 9110 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260616 19:58:17.300314 9122 log.cc:927] Time spent T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.077s user 0.000s sys 0.001s
I20260616 19:58:17.341981 9068 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:17.343708 9067 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.328773 9022 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260616 19:58:17.415653 9129 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:17.344974 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 85711us
I20260616 19:58:17.345290 9027 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:17.345757 9090 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260616 19:58:17.346472 9098 consensus_meta.cc:287] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.509s user 0.009s sys 0.000s
I20260616 19:58:17.418540 7935 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.418995 9098 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:17.418649 9120 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.346836 9086 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.418478 9027 consensus_queue.cc:1059] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260616 19:58:17.335592 9041 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.353230 8117 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:17.323136 9118 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:17.356505 8084 leader_election.cc:403] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260616 19:58:17.353986 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.422734 9068 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:17.364125 8040 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:17.419907 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.332212 8916 consensus_queue.cc:1059] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260616 19:58:17.334291 8898 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260616 19:58:17.358402 8973 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:17.433539 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 105511us
I20260616 19:58:17.462198 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.465031 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.466136 8040 raft_consensus.cc:2393] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:17.472510 9068 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.475481 8916 raft_consensus.cc:3055] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:17.449046 8934 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3.
I20260616 19:58:17.477898 9102 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:17.339265 9105 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.475996 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.510763 8042 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:17.478622 9002 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.479219 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.483530 9110 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.476003 8913 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.481575 9045 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.487013 8117 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:17.425545 9081 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.518630 9002 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.491564 8916 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:17.520242 9118 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.520648 9045 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.496111 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.521420 8043 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:17.522091 8913 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.522213 9118 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.522373 9003 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.498075 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.507470 8973 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.525267 8973 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:17.526655 8039 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260616 19:58:17.508705 9131 log.cc:927] Time spent T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260616 19:58:17.508869 9102 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.520128 8916 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.489449 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.535610 8036 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:17.491751 9106 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:17.529501 9035 consensus_meta.cc:287] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.630s user 0.009s sys 0.002s
I20260616 19:58:17.506134 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:17.548730 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 289248us
I20260616 19:58:17.538937 8043 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:17.543737 7966 raft_consensus.cc:1240] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:17.543949 9068 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:17.547288 8113 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:17.546536 9003 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.586042 8040 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:17.547730 9035 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.581219 9097 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.625912 9129 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:17.628652 9102 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.615072 9041 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.634019 8116 raft_consensus.cc:1240] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:17.634853 7868 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55744 (request call id 127) took 1873 ms. Trace:
W20260616 19:58:17.642452 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 369662us
I20260616 19:58:17.635062 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.644591 7934 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.641341 7868 rpcz_store.cc:276] 0616 19:58:15.824883 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:15.830469 (+ 5586us) service_pool.cc:224] Handling call
0616 19:58:17.634789 (+1804320us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:16.705139 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:16.830545 (+125406us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:16.830608 (+ 63us) write_op.cc:435] Acquired schema lock
0616 19:58:16.830637 (+ 29us) tablet.cc:662] Decoding operations
0616 19:58:16.831089 (+ 452us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:16.831141 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:16.831178 (+ 37us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:16.831371 (+ 193us) tablet.cc:701] Row locks acquired
0616 19:58:16.831395 (+ 24us) write_op.cc:260] PREPARE: finished
0616 19:58:16.831536 (+ 141us) write_op.cc:270] Start()
0616 19:58:16.831731 (+ 195us) write_op.cc:276] Timestamp: P: 1781639896831487 usec, L: 0
0616 19:58:16.831764 (+ 33us) op_driver.cc:348] REPLICATION: starting
0616 19:58:16.832496 (+ 732us) log.cc:844] Serialized 540 byte log entry
0616 19:58:17.580376 (+747880us) op_driver.cc:464] REPLICATION: finished
0616 19:58:17.630776 (+ 50400us) write_op.cc:301] APPLY: starting
0616 19:58:17.630855 (+ 79us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:17.630960 (+ 105us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:17.631044 (+ 84us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:17.631236 (+ 192us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:17.631377 (+ 141us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:17.631427 (+ 50us) write_op.cc:312] APPLY: finished
0616 19:58:17.631586 (+ 159us) log.cc:844] Serialized 22 byte log entry
0616 19:58:17.631742 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:17.631808 (+ 66us) write_op.cc:454] Released schema lock
0616 19:58:17.631838 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":49536,"apply.run_cpu_time_us":1293,"apply.run_wall_time_us":1292,"num_ops":1,"prepare.queue_time_us":126480,"prepare.run_cpu_time_us":3185,"prepare.run_wall_time_us":135838,"replication_time_us":737953,"threads_started":1}]]}
W20260616 19:58:17.650661 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 339192us
I20260616 19:58:17.650847 9080 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:17.651511 9080 raft_consensus.cc:697] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.652948 9080 consensus_queue.cc:237] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.661350 9067 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:17.662338 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:17.664739 8011 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.667706 8112 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:17.669358 8118 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:17.669414 8011 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:17.663968 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:17.674157 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:17.633632 9097 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.633405 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 129) took 2839 ms. Trace:
I20260616 19:58:17.681968 8011 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.683204 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:17.685269 9138 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:17.689268 9138 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:17.689415 9035 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.689700 9138 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:17.687196 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:17.628952 9064 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.674219 9067 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:17.690903 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:17.699939 8111 raft_consensus.cc:2410] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:58:17.700840 8898 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:17.701059 9110 consensus_queue.cc:1059] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:17.701982 9110 raft_consensus.cc:3055] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:17.702307 9110 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.703151 9110 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:17.704128 9064 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.704921 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:17.690541 9125 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:17.698571 9097 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:17.698298 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.648253 9018 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260616 19:58:17.708781 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:17.685863 9068 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260616 19:58:17.710148 9064 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:17.713022 8118 raft_consensus.cc:2393] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:17.713212 7934 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.715407 9106 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:17.724591 8116 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:17.725924 8116 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:17.735028 8119 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:17.712239 9097 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.713755 9067 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:17.738528 7934 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:17.739908 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 320053us
I20260616 19:58:17.740703 7935 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.744059 7935 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:17.746955 7935 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.716754 9064 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.746496 9129 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:17.756687 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.746474 9137 consensus_queue.cc:1059] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260616 19:58:17.750222 9043 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:17.746892 9106 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:17.758821 9129 raft_consensus.cc:697] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.761974 9129 consensus_queue.cc:237] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.745488 9018 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260616 19:58:17.769820 9035 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.758545 9143 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.745749 9067 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.775962 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:17.778261 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:17.754695 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.784955 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.775385 9043 raft_consensus.cc:697] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:17.793179 7967 raft_consensus.cc:2393] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:17.783700 7969 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:17.790030 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.783538 8009 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:17.796782 7964 raft_consensus.cc:1240] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:17.792037 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:17.779197 9035 consensus_queue.cc:1059] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260616 19:58:17.793936 9018 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:17.908169 8040 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:17.910615 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.912231 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:17.919979 8043 raft_consensus.cc:2393] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:17.913952 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:17.904893 9142 log.cc:927] Time spent T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.108s user 0.000s sys 0.001s
I20260616 19:58:17.800275 9141 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:17.801786 8916 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:17.904848 9061 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:17.941084 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 149150us
I20260616 19:58:17.942178 7935 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.804693 9125 consensus_queue.cc:1059] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.122s
I20260616 19:58:17.798233 9068 consensus_queue.cc:1059] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.124s
I20260616 19:58:17.954430 8011 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:17.959774 8036 raft_consensus.cc:1240] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:17.965368 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:17.966737 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:17.971006 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:17.923149 9098 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:17.972186 8036 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:17.985785 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.973385 8084 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.951225 9035 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:17.992630 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:17.994045 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:17.990672 8043 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:17.991432 8913 consensus_queue.cc:1059] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:17.951686 9141 raft_consensus.cc:697] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:17.989674 7965 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:17.975419 9080 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:18.000845 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:18.001226 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:17.974920 9097 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:18.001493 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.005167 9133 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:18.005314 9140 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.128s user 0.000s sys 0.001s
I20260616 19:58:18.011344 8084 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:17.989949 9137 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:17.951871 9061 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:17.986922 9064 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.046826 9067 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:17.987414 9098 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:18.048843 9139 log.cc:927] Time spent T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.074s user 0.000s sys 0.000s
I20260616 19:58:18.050419 9150 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:18.006908 9035 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:18.051139 9150 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:18.051690 9035 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.053383 9061 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:18.053524 9108 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:18.050490 9098 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:17.953879 9043 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:18.055847 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 44603.8us
I20260616 19:58:18.018052 9088 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:18.012925 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.048867 9067 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:18.056622 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 64748.5us
W20260616 19:58:18.062083 9144 log.cc:927] Time spent T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20260616 19:58:18.057830 8934 raft_consensus.cc:3055] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:18.023960 9133 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.065131 9067 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.055029 9137 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.071599 9133 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.073199 8084 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:18.068187 9141 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:18.053376 9150 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.069741 8934 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:18.077322 9150 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:18.048478 9023 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:18.085477 9023 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.087682 9023 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:18.072362 9137 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.097118 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:18.098778 8084 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:18.074436 9067 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.137656 8913 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:18.193485 8913 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:18.144855 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:18.138166 8084 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:18.197710 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 59644us
W20260616 19:58:18.214313 9148 log.cc:927] Time spent T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.114s user 0.000s sys 0.000s
I20260616 19:58:18.094163 9098 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:18.194535 8913 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.230763 9145 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:18.232334 8934 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.233979 8934 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:18.237177 8041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 574) took 1050 ms. Trace:
I20260616 19:58:18.237723 8041 rpcz_store.cc:276] 0616 19:58:17.187062 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.269422 (+ 82360us) service_pool.cc:224] Handling call
0616 19:58:17.269723 (+ 301us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:17.355735 (+ 86012us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:17.355779 (+ 44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:17.367715 (+ 11936us) log.cc:844] Serialized 26 byte log entry
0616 19:58:17.684719 (+317004us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:17.684757 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:17.684818 (+ 61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:18.236877 (+552059us) raft_consensus.cc:1685] finished
0616 19:58:18.236908 (+ 31us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:18.237096 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59616,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6495,"thread_start_us":306220,"threads_started":2,"wal-append.queue_time_us":490396}
I20260616 19:58:18.126609 9064 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:18.229198 9098 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:18.256783 9138 consensus_meta.cc:287] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.567s user 0.007s sys 0.005s
I20260616 19:58:18.257252 9138 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.258864 8042 raft_consensus.cc:1240] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
W20260616 19:58:18.240592 9014 consensus_queue.cc:1175] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260616 19:58:18.248803 9023 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:18.260085 9138 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:18.271330 8117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s user 0.018s sys 0.031s
W20260616 19:58:18.281525 8117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s user 0.018s sys 0.031s
I20260616 19:58:18.285372 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:18.286260 8110 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:18.255177 9098 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:18.271520 8119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s user 0.012s sys 0.036s
W20260616 19:58:18.291224 8119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s user 0.012s sys 0.036s
I20260616 19:58:18.275802 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:18.292366 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:18.293681 9152 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.180s user 0.000s sys 0.002s
I20260616 19:58:18.294018 9049 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:18.294447 9049 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:18.249063 9064 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.313967 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:18.291687 9034 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:18.282473 9132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s user 0.026s sys 0.040s
I20260616 19:58:18.295112 9022 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:18.313551 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:18.329524 9049 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.339416 9064 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.287897 9023 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:18.341015 9132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s user 0.026s sys 0.040s
I20260616 19:58:18.340797 8116 raft_consensus.cc:1240] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:17.682583 7864 rpcz_store.cc:276] 0616 19:58:14.794016 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:14.794206 (+ 190us) service_pool.cc:224] Handling call
0616 19:58:15.586591 (+792385us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:17.633315 (+2046724us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:16.580574 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:16.587814 (+ 7240us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:16.587863 (+ 49us) write_op.cc:435] Acquired schema lock
0616 19:58:16.587892 (+ 29us) tablet.cc:662] Decoding operations
0616 19:58:16.588437 (+ 545us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:16.588520 (+ 83us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:16.588568 (+ 48us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:16.588826 (+ 258us) tablet.cc:701] Row locks acquired
0616 19:58:16.588850 (+ 24us) write_op.cc:260] PREPARE: finished
0616 19:58:16.588977 (+ 127us) write_op.cc:270] Start()
0616 19:58:16.589092 (+ 115us) write_op.cc:276] Timestamp: P: 1781639896588934 usec, L: 0
0616 19:58:16.589115 (+ 23us) op_driver.cc:348] REPLICATION: starting
0616 19:58:16.589733 (+ 618us) log.cc:844] Serialized 540 byte log entry
0616 19:58:16.896577 (+306844us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b18002be220. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:17.242833 (+346256us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b18002be220. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:17.243137 (+ 304us) op_driver.cc:464] REPLICATION: finished
0616 19:58:17.628676 (+385539us) write_op.cc:301] APPLY: starting
0616 19:58:17.628780 (+ 104us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:17.628905 (+ 125us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:17.628936 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:17.629253 (+ 317us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:17.629446 (+ 193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:17.629539 (+ 93us) write_op.cc:312] APPLY: finished
0616 19:58:17.629774 (+ 235us) log.cc:844] Serialized 22 byte log entry
0616 19:58:17.629976 (+ 202us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:17.630071 (+ 95us) write_op.cc:454] Released schema lock
0616 19:58:17.630115 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:17.630447 (+ 332us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:17.630475 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":52175,"reported_tablets":1,"spinlock_wait_cycles":342913792,"child_traces":[["op",{"apply.queue_time_us":385255,"apply.run_cpu_time_us":1799,"apply.run_wall_time_us":1800,"mutex_wait_us":9614,"num_ops":1,"prepare.queue_time_us":185885,"prepare.run_cpu_time_us":30238,"prepare.run_wall_time_us":129100,"raft.queue_time_us":675838,"raft.run_cpu_time_us":29588,"raft.run_wall_time_us":309041,"replication_time_us":653895,"spinlock_wait_cycles":10277632,"thread_start_us":512897,"threads_started":4,"wal-append.queue_time_us":91182}]]}
I20260616 19:58:18.373227 9023 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:18.374686 8115 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:18.357470 9049 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:18.398754 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:18.403537 8041 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:18.385468 9064 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:18.396332 8786 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s user 0.025s sys 0.030s
I20260616 19:58:18.410785 8041 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260616 19:58:18.412006 8786 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s user 0.025s sys 0.030s
W20260616 19:58:18.385006 9129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s user 0.014s sys 0.042s
I20260616 19:58:18.423998 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:18.425770 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:18.439822 8039 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:18.443449 8036 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:18.447609 8039 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:18.453402 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:18.407886 9064 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.462241 8045 raft_consensus.cc:2393] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:58:18.462710 9160 log.cc:927] Time spent T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.077s user 0.000s sys 0.001s
W20260616 19:58:18.460637 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 710818us
I20260616 19:58:18.459188 9106 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:18.465921 9106 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:18.466800 9081 raft_consensus.cc:3055] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:18.467229 9081 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:18.477411 9064 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.480952 8039 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260616 19:58:18.433305 9129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s user 0.014s sys 0.043s
W20260616 19:58:18.488473 8011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.009s sys 0.016s
W20260616 19:58:18.490118 8011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s user 0.009s sys 0.016s
W20260616 19:58:18.492005 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 537644us
I20260616 19:58:18.506402 8009 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:18.507495 7969 raft_consensus.cc:1275] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:18.510161 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.511139 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.509032 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:18.513113 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.513664 9064 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:18.514163 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.514340 9064 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.512094 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.516535 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:18.520072 7964 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:18.523138 7934 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:18.523950 9128 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:18.525017 9067 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:18.525171 9088 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:18.525429 9067 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:18.533151 7934 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:18.525887 9067 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:18.573534 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:18.576303 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:18.572707 8044 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s user 0.009s sys 0.028s
W20260616 19:58:18.580181 8044 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s user 0.009s sys 0.028s
I20260616 19:58:18.534858 8916 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:18.526895 8927 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:18.598304 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:18.598944 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 399840us
I20260616 19:58:18.538576 9080 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
W20260616 19:58:18.593927 7965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.007s sys 0.026s
I20260616 19:58:18.525903 9088 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.494352 9081 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.599717 8913 consensus_queue.cc:1059] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:18.535406 9106 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:18.580026 9143 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:18.601115 8118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s user 0.013s sys 0.011s
W20260616 19:58:18.772857 7965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s user 0.007s sys 0.030s
W20260616 19:58:18.773716 8118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s user 0.016s sys 0.011s
W20260616 19:58:18.713862 8040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s user 0.029s sys 0.026s
W20260616 19:58:18.780439 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:18.697065 9035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s user 0.013s sys 0.028s
W20260616 19:58:18.778808 8040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s user 0.030s sys 0.026s
I20260616 19:58:18.793879 8118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 620) took 1120 ms. Trace:
I20260616 19:58:18.602962 8927 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.592242 9034 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:18.794344 8118 rpcz_store.cc:276] 0616 19:58:17.661067 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.714525 (+ 53458us) service_pool.cc:224] Handling call
0616 19:58:17.714835 (+ 310us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:17.715061 (+ 226us) raft_consensus.cc:1534] Early marking committed up to index 2
0616 19:58:18.011648 (+296587us) log.cc:844] Serialized 14 byte log entry
0616 19:58:18.780702 (+769054us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:58:18.780772 (+ 70us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:18.780831 (+ 59us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:18.780934 (+ 103us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:18.781095 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6385,"prepare.queue_time_us":347295,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":40,"thread_start_us":1044526,"threads_started":2}
I20260616 19:58:18.601239 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:18.589829 9105 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s user 0.004s sys 0.044s
W20260616 19:58:18.792527 9035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s user 0.018s sys 0.029s
I20260616 19:58:18.986616 9088 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.986904 9041 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:18.987751 9041 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:18.773188 9064 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:18.808327 7966 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:18.699074 9169 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:18.600337 7964 raft_consensus.cc:1275] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:18.809013 8927 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:18.739682 9164 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:18.738749 9168 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:18.622529 9145 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:18.598335 9108 consensus_queue.cc:1059] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.209s
I20260616 19:58:18.742954 9061 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260616 19:58:18.784467 9143 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.025274 7866 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260616 19:58:18.836336 9159 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.334s user 0.000s sys 0.001s
W20260616 19:58:18.840286 9156 log.cc:927] Time spent T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.305s user 0.000s sys 0.001s
I20260616 19:58:18.840741 7968 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:18.841640 9004 raft_consensus.cc:3055] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:18.828663 9081 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:18.844415 9141 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:18.848242 9023 raft_consensus.cc:3055] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:18.780516 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:18.601740 8916 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:18.647341 9098 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:18.852385 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 253448us
I20260616 19:58:18.862445 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:18.644282 9165 consensus_queue.cc:1059] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.217s
W20260616 19:58:18.868327 9105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.071s user 0.013s sys 0.044s
I20260616 19:58:18.849038 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:18.858582 9068 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s user 0.001s sys 0.026s
I20260616 19:58:18.987465 9166 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:19.015159 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 473556us
I20260616 19:58:18.989486 9041 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:18.836192 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:19.027690 7866 catalog_manager.cc:5696] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260616 19:58:19.034958 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:19.074397 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:19.012423 9064 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.270870 9064 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.016222 9035 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.080992 8042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 844) took 1165 ms. Trace:
I20260616 19:58:19.052690 9004 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:19.127029 8114 raft_consensus.cc:1240] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:19.127345 8116 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:19.082060 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:19.014593 9169 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.079870 9023 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:19.280637 8037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51870 (request call id 846) took 1370 ms (client timeout 1610 ms). Trace:
I20260616 19:58:19.281061 9023 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.281668 8916 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.282039 9169 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.281517 9004 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.282526 9023 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:19.282850 9004 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:19.122320 9068 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.132s user 0.001s sys 0.027s
I20260616 19:58:19.284706 9068 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.081806 9141 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:19.131376 8820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s user 0.004s sys 0.008s
I20260616 19:58:19.030840 9168 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.130152 9166 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:19.213409 9176 log.cc:927] Time spent T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
W20260616 19:58:19.129263 8084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s user 0.009s sys 0.025s
W20260616 19:58:19.292479 8084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s user 0.018s sys 0.049s
W20260616 19:58:19.292785 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 693190us
I20260616 19:58:19.293723 8084 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:19.255090 9102 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:19.291931 9068 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.297125 9102 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.297720 9141 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.257028 9199 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.213933 9145 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.258052 9127 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.266834 7866 catalog_manager.cc:5696] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:19.268342 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:19.257658 9162 log.cc:927] Time spent T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.176s user 0.005s sys 0.000s
I20260616 19:58:19.269209 9183 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:19.272063 8042 rpcz_store.cc:276] 0616 19:58:17.909813 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.910614 (+ 801us) service_pool.cc:224] Handling call
0616 19:58:17.910928 (+ 314us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:18.259383 (+348455us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.068712 (+809329us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400367c54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:19.075343 (+ 6631us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799588096}
W20260616 19:58:19.272125 9080 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s user 0.010s sys 0.015s
I20260616 19:58:19.277664 9035 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.029852 9164 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:19.291935 8820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s user 0.004s sys 0.009s
W20260616 19:58:19.281080 8037 rpcz_store.cc:269] 0616 19:58:17.910501 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.912791 (+ 2290us) service_pool.cc:224] Handling call
0616 19:58:19.068690 (+1155899us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800011498. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:58:19.280553 (+211863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24240896}
I20260616 19:58:19.304734 8084 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:19.309540 9199 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:19.310050 9102 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:19.295634 9178 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.208s user 0.000s sys 0.007s
W20260616 19:58:19.327245 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 59000.9us
I20260616 19:58:19.287166 9041 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:19.310698 7866 catalog_manager.cc:5696] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:19.328603 8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52474 (request call id 2) took 1550 ms. Trace:
I20260616 19:58:19.328830 8818 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 LEARNER]: Advancing to term 4
I20260616 19:58:19.329021 8820 rpcz_store.cc:276] 0616 19:58:17.793684 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.800715 (+ 7031us) service_pool.cc:224] Handling call
0616 19:58:17.921528 (+120813us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:18.460594 (+539066us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:18.461600 (+ 1006us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:18.588604 (+127004us) log.cc:844] Serialized 26 byte log entry
0616 19:58:19.328168 (+739564us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:19.328226 (+ 58us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.328327 (+ 101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:19.328369 (+ 42us) raft_consensus.cc:1685] finished
0616 19:58:19.328385 (+ 16us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:19.328527 (+ 142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17050,"raft.queue_time_us":123091,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":505,"thread_start_us":1191939,"threads_started":2,"wal-append.queue_time_us":662371}
I20260616 19:58:19.297741 9168 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.303798 8946 consensus_queue.cc:1048] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
W20260616 19:58:19.315073 9067 consensus_meta.cc:287] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.726s user 0.010s sys 0.001s
I20260616 19:58:19.315510 9183 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.310530 9127 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.298524 9068 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.318411 9165 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.325400 8084 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:19.317447 9080 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s user 0.011s sys 0.015s
I20260616 19:58:19.328615 7866 catalog_manager.cc:5696] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:19.319362 9191 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.338557 9098 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.323664 9022 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.324015 9141 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.338065 9189 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.347311 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.338485 7866 catalog_manager.cc:5696] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:19.333170 8011 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.349089 8036 raft_consensus.cc:1275] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.351227 8011 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.312245 9108 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:19.332370 9164 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.347931 8042 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:19.350677 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.332777 9067 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.339062 9127 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.352834 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.370111 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.371127 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.348764 7866 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:19.376384 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 834) took 1589 ms. Trace:
I20260616 19:58:19.339886 9080 raft_consensus.cc:697] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:19.379047 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.390175 8117 raft_consensus.cc:2410] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:19.382328 8011 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.352500 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.376965 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.376705 7966 rpcz_store.cc:276] 0616 19:58:17.780644 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.784981 (+ 4337us) service_pool.cc:224] Handling call
0616 19:58:17.793653 (+ 8672us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:19.009247 (+1215594us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.370027 (+360780us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540024dbd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:19.370176 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2329065728}
I20260616 19:58:19.339306 9165 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.391885 8043 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:19.334389 9102 consensus_queue.cc:1059] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260616 19:58:19.353065 9141 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.410441 8036 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:19.410742 8011 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.394922 8043 raft_consensus.cc:1275] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.380144 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.379771 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:19.395958 9193 log.cc:927] Time spent T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.124s user 0.000s sys 0.001s
I20260616 19:58:19.378847 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.420634 9080 consensus_queue.cc:237] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.421605 8113 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:19.396679 9109 consensus_queue.cc:1059] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260616 19:58:19.423246 9109 raft_consensus.cc:3055] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:19.423689 9109 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:19.338714 9191 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.413894 9141 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.380437 9143 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.414937 9081 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:19.417302 9035 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.352279 9190 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:19.370275 9196 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:19.400802 9164 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:19.437803 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.438987 8119 raft_consensus.cc:2410] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:19.439663 8927 raft_consensus.cc:3055] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:19.439993 8927 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:19.442808 8927 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.444252 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.368268 9098 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:19.413805 9102 raft_consensus.cc:3055] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:19.426023 8011 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.402611 9179 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:19.444994 8117 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:19.432832 9127 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.375628 9189 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.449505 8011 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.451367 8011 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.454329 8011 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.446309 9102 raft_consensus.cc:740] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:19.450440 9164 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.468963 8118 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:19.451143 9081 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.463554 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.499647 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.469091 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.500473 8116 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:19.500892 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.501359 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.457490 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.455387 9141 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260616 19:58:19.514813 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 57524.7us
I20260616 19:58:19.433995 9190 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.515578 8110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 634) took 1743 ms. Trace:
I20260616 19:58:19.515975 8011 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.516922 9196 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.517081 9190 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.515975 8110 rpcz_store.cc:276] 0616 19:58:17.771979 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:18.319390 (+547411us) service_pool.cc:224] Handling call
0616 19:58:18.326242 (+ 6852us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:18.345099 (+ 18857us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:18.345136 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:18.356129 (+ 10993us) log.cc:844] Serialized 26 byte log entry
0616 19:58:19.419227 (+1063098us) spinlock_profiling.cc:243] Waited 889 us on lock 0x7b1800350bc0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c216587 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe86ee7e5ab 00007fe86ee840c3 00007fe86ee84471 00007fe86f2a0def
0616 19:58:19.450795 (+ 31568us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:19.450948 (+ 153us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.451320 (+ 372us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:19.515314 (+ 63994us) raft_consensus.cc:1685] finished
0616 19:58:19.515332 (+ 18us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:19.515524 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7966,"spinlock_wait_cycles":2045312,"thread_start_us":31372,"threads_started":1,"wal-append.queue_time_us":1102519}
I20260616 19:58:19.517514 8011 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.518162 9196 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.500344 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:19.456430 9127 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.524039 9198 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260616 19:58:19.525560 7963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 539) took 2326 ms (client timeout 2148 ms). Trace:
W20260616 19:58:19.529938 7963 rpcz_store.cc:269] 0616 19:58:17.187807 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.188486 (+ 679us) service_pool.cc:224] Handling call
0616 19:58:19.513574 (+2325088us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b5400002dd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:19.513748 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376674176}
I20260616 19:58:19.519291 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.519749 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.530964 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 633) took 1785 ms. Trace:
I20260616 19:58:19.531327 8114 rpcz_store.cc:276] 0616 19:58:17.745486 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:18.317384 (+571898us) service_pool.cc:224] Handling call
0616 19:58:18.317728 (+ 344us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:19.273080 (+955352us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.530730 (+257650us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400438354. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:19.530914 (+ 184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860502400}
I20260616 19:58:19.532405 8011 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.519115 9179 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.535660 9198 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:19.535794 9179 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:19.525280 9188 log.cc:927] Time spent T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.268s user 0.005s sys 0.000s
I20260616 19:58:19.535259 8818 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260616 19:58:19.520560 9164 consensus_queue.cc:1059] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260616 19:58:19.539052 9183 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.535842 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.548854 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.548453 9200 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.453634 9109 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.568594 8037 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.571195 9166 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.571762 8037 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.572849 9199 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:19.574209 8037 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:19.575963 8037 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:19.576220 8110 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.576574 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.540421 9198 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.542426 9141 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.551649 8818 pending_rounds.cc:85] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Aborting all ops after (but not including) 2
I20260616 19:58:19.557372 9133 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.542181 9068 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.544900 9168 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.542064 9102 consensus_queue.cc:260] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.527120 9127 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.566187 9200 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.569252 9189 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.569574 9109 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:19.535238 9165 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.578573 8116 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.549860 9067 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:19.582263 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 54438.7us
I20260616 19:58:19.588049 8117 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.590061 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.591495 9191 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.584446 8818 pending_rounds.cc:107] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260616 19:58:19.601374 8913 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.601733 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.602290 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.601415 8044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 577) took 1643 ms. Trace:
I20260616 19:58:19.601887 8913 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:19.603008 8117 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:19.603068 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:19.602952 8044 rpcz_store.cc:276] 0616 19:58:17.958143 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.958293 (+ 150us) service_pool.cc:224] Handling call
0616 19:58:17.958734 (+ 441us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:18.581923 (+623189us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:18.581967 (+ 44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:18.595786 (+ 13819us) log.cc:844] Serialized 26 byte log entry
0616 19:58:19.035609 (+439823us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:19.035662 (+ 53us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.035754 (+ 92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:19.428858 (+393104us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:19.599772 (+170914us) raft_consensus.cc:1685] finished
0616 19:58:19.599804 (+ 32us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:19.601359 (+ 1555us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25611,"raft.queue_time_us":65316,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":9593,"spinlock_wait_cycles":28480256,"thread_start_us":987423,"threads_started":2,"wal-append.queue_time_us":629501}
I20260616 19:58:19.606158 8112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 635) took 1078 ms. Trace:
I20260616 19:58:19.606842 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.606118 9200 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.607461 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.606683 8112 rpcz_store.cc:276] 0616 19:58:18.528034 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.574759 (+1046725us) service_pool.cc:224] Handling call
0616 19:58:19.575005 (+ 246us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:19.575180 (+ 175us) raft_consensus.cc:1534] Early marking committed up to index 2
0616 19:58:19.575433 (+ 253us) log.cc:844] Serialized 14 byte log entry
0616 19:58:19.605859 (+ 30426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:58:19.605910 (+ 51us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:19.605950 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.606013 (+ 63us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:19.606109 (+ 96us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":509,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":825,"thread_start_us":25588,"threads_started":1,"wal-append.queue_time_us":29266}
I20260616 19:58:19.600788 8818 raft_consensus.cc:2889] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:19.610409 8913 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.617645 9127 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.618112 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:19.587546 9141 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.587491 9133 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.620297 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.588583 9068 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.590250 9168 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.608820 9102 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:19.644246 9141 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.649428 9190 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.649991 9190 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.653347 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 848) took 1145 ms. Trace:
I20260616 19:58:19.655323 8110 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.626120 8116 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:19.584444 9198 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.657117 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.657987 8117 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.653754 7968 rpcz_store.cc:276] 0616 19:58:18.507583 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:18.516291 (+ 8708us) service_pool.cc:224] Handling call
0616 19:58:19.653160 (+1136869us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54002fc954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:19.653290 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743861632}
I20260616 19:58:19.667706 8110 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:19.669901 9198 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.670439 9168 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.671629 8110 raft_consensus.cc:3055] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:19.671977 8110 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:19.652019 9190 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.635835 8820 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:19.672739 8110 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.676134 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.680027 8110 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:19.677781 8116 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:19.678536 8008 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.684682 9132 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:19.685065 9132 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.685438 9132 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:19.678876 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.677351 9207 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.687621 8111 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:19.688210 9207 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:58:19.690145 9180 log.cc:927] Time spent T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.538s user 0.008s sys 0.003s
I20260616 19:58:19.695713 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 855) took 1186 ms. Trace:
I20260616 19:58:19.696184 7962 rpcz_store.cc:276] 0616 19:58:18.509323 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:18.516098 (+ 6775us) service_pool.cc:224] Handling call
0616 19:58:18.591884 (+ 75786us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:19.108284 (+516400us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:19.123138 (+ 14854us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:58:19.130861 (+ 7723us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:19.140400 (+ 9539us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.695452 (+555052us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54003f7d54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:19.695510 (+ 58us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:19.695653 (+ 143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125219200}
I20260616 19:58:19.707314 8037 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:19.707979 8009 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.708990 9133 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.709549 9133 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:19.709919 9133 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:19.710258 9133 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:19.715042 8119 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.719444 9064 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:19.729456 9194 log.cc:927] Time spent T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.471s user 0.004s sys 0.004s
I20260616 19:58:19.732703 8040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 579) took 1772 ms. Trace:
I20260616 19:58:19.733392 8009 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.734524 8110 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260616 19:58:19.734972 8110 pending_rounds.cc:85] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 1
I20260616 19:58:19.735184 8110 pending_rounds.cc:107] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260616 19:58:19.735392 8110 raft_consensus.cc:2889] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:19.733048 8040 rpcz_store.cc:276] 0616 19:58:17.960421 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.960568 (+ 147us) service_pool.cc:224] Handling call
0616 19:58:17.960862 (+ 294us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:18.793127 (+832265us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:18.793174 (+ 47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:18.793718 (+ 544us) log.cc:844] Serialized 26 byte log entry
0616 19:58:19.120485 (+326767us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:19.120540 (+ 55us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.120657 (+ 117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:19.720545 (+599888us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:19.730651 (+ 10106us) raft_consensus.cc:1685] finished
0616 19:58:19.730692 (+ 41us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:19.732638 (+ 1946us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9354,"raft.queue_time_us":89130,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3977,"spinlock_wait_cycles":30546176,"thread_start_us":1091400,"threads_started":2,"wal-append.queue_time_us":421826}
I20260616 19:58:19.736528 9198 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.736662 8009 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.736950 9198 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:19.737352 9198 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.737715 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.738592 8009 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.739042 9168 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:19.737771 8110 raft_consensus.cc:1240] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:19.740952 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.737694 9169 raft_consensus.cc:3055] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:19.741307 9064 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.745210 9064 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:19.745556 9064 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.746286 9125 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:19.746655 9125 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:19.746309 8009 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.749660 8110 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:19.749637 9127 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:19.750391 9127 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.744557 9141 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260616 19:58:19.750774 9127 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:19.750927 8009 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.751513 9138 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.751904 9138 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:19.752256 9138 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.753791 9168 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:19.741201 9121 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.754482 8009 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.754649 9168 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.755127 9168 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:19.755847 8009 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.755321 9141 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.748652 9169 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:19.770443 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.747502 9125 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.780762 9125 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.780980 8118 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:19.755040 9121 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.759719 9088 consensus_queue.cc:1059] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:19.757849 9200 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.761543 9110 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.757712 9141 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:19.788849 9141 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.788278 9200 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:19.805284 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.805828 8118 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:19.806732 8009 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:19.800046 9200 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.810927 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 562) took 1822 ms. Trace:
I20260616 19:58:19.808202 9198 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.799031 9207 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.811237 7967 rpcz_store.cc:276] 0616 19:58:17.988451 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.995039 (+ 6588us) service_pool.cc:224] Handling call
0616 19:58:17.995389 (+ 350us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:18.074733 (+ 79344us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:18.074793 (+ 60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:18.723993 (+649200us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b2c000328a8. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86dbdec71 00007fe86dbee5d2 00007fe86f320de1 00007fe86f2e581a 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22
0616 19:58:18.782000 (+ 58007us) log.cc:844] Serialized 26 byte log entry
0616 19:58:19.105694 (+323694us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:19.125933 (+ 20239us) log.cc:844] Serialized 14 byte log entry
0616 19:58:19.669264 (+543331us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b180005adc0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c201be6 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c1a7e1a 00007fe8758b35d9
0616 19:58:19.702891 (+ 33627us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b180005adc0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c201be6 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01
0616 19:58:19.704569 (+ 1678us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b180005adc0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:19.787489 (+ 82920us) spinlock_profiling.cc:243] Waited 8.56 ms on lock 0x7b180005adc0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c1a7e1a 00007fe8758b35d9 00007fe86c0661f0 00007fe86c230b24 00007fe86c230a95 00007fe86f2dd440 00007fe86f2efaf6 00007fe86f300667 00007fe86f3005e8
0616 19:58:19.787664 (+ 175us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.810695 (+ 23031us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b54003f7854. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:19.810731 (+ 36us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:19.810763 (+ 32us) raft_consensus.cc:1685] finished
0616 19:58:19.810772 (+ 9us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:19.810881 (+ 109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5146,"prepare.queue_time_us":23349,"prepare.run_cpu_time_us":9133,"prepare.run_wall_time_us":77894,"raft.queue_time_us":662149,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":17454,"spinlock_wait_cycles":239213312,"thread_start_us":272142,"threads_started":1,"wal-append.queue_time_us":449899}
I20260616 19:58:19.807530 9068 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.819573 9185 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.822870 9143 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:19.822642 9168 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.812939 9088 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.823385 9041 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.824448 9088 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:19.822561 9185 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.826210 9106 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.826757 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.828195 8039 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:19.825350 9080 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:19.828599 9106 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.828949 9080 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.830547 9106 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.830876 9080 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.825947 9041 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.832960 8927 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
W20260616 19:58:19.836835 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:19.837262 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:19.837582 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:19.838088 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:19.840194 7963 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260616 19:58:19.840509 7963 pending_rounds.cc:85] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:19.840744 7963 pending_rounds.cc:107] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260616 19:58:19.840631 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.840983 7963 raft_consensus.cc:2889] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:19.826005 9088 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.842614 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:19.825487 9027 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.825681 9004 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:19.850265 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:19.845646 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:19.852538 8112 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.854872 8913 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:19.855329 8913 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.855639 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.856390 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.858415 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.860069 8119 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:19.860680 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.848483 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.866662 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.861217 7963 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:19.869210 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:19.841830 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:19.872538 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.882246 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.883003 8040 raft_consensus.cc:2393] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:19.883738 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.873200 8913 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.823349 9143 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.876921 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.841825 9129 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.905443 8085 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:19.912462 8927 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:19.912963 8927 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:19.913384 8927 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:19.906862 8084 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.907384 8913 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:19.854444 9198 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:19.928499 8913 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.930847 8913 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:19.931344 8913 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:19.921298 7934 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:19.929438 8084 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:19.865089 9105 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:19.938333 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.941079 8084 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:19.945688 9133 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.874735 9043 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:19.951530 8084 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:19.902019 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.900653 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:19.901091 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:19.958881 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.921224 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:19.953095 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:19.927181 9003 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260616 19:58:19.927629 9036 consensus_queue.cc:1059] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260616 19:58:19.991756 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:19.953712 9068 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:19.953919 9061 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:19.969764 9145 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:19.971418 9132 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:19.939926 9186 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.899381 9080 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:19.874635 9182 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260616 19:58:19.867727 9027 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.871145 9004 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.875033 9138 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.948817 9109 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:19.930162 9121 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.929556 9129 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.987790 9120 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:19.981868 9196 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:19.956631 9105 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.013286 8084 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:19.976274 8927 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.058019 8913 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:20.121047 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 97168.2us
I20260616 19:58:19.981222 8045 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:20.032910 9080 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:19.978189 9195 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260616 19:58:20.123088 8927 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:19.985873 9033 consensus_queue.cc:1048] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260616 19:58:19.978530 9043 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.064643 7963 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
W20260616 19:58:20.130527 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 209280us
I20260616 19:58:20.119836 7935 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.136930 8927 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260616 19:58:20.137738 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 124576us
W20260616 19:58:20.138458 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.150203 8085 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.151558 8913 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:20.035642 8111 raft_consensus.cc:2393] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:20.129419 8112 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:20.146565 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:20.152523 8084 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.155453 9132 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:20.049988 9036 raft_consensus.cc:3055] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:20.156308 9036 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:20.156392 9132 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:20.157168 9036 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.157256 9132 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.158416 9036 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:20.161751 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.162057 9132 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.162982 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.167760 9080 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.169628 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.176980 7961 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:20.178222 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.179538 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:19.985587 9198 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.127s
W20260616 19:58:20.193941 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.096678 9168 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.193200 8927 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:20.037091 9145 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:20.202231 7970 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:20.195793 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.005313 9068 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.030730 9127 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.217347 9127 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.218770 9035 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:20.218871 9064 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.204768 9097 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260616 19:58:20.224524 9027 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.225045 9027 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:20.227421 9105 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.200410 9185 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.203387 9145 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:20.230072 9018 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:20.230590 9018 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:20.231906 7967 raft_consensus.cc:2410] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260616 19:58:20.205757 9120 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260616 19:58:20.237763 9043 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.240715 9221 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:20.241278 9221 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.243623 9221 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:20.197089 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 48938.4us
I20260616 19:58:20.128669 9107 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:20.150733 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.142747 9195 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:20.205368 7961 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:20.206827 9003 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:20.258044 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.258733 9195 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.259466 9169 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.259943 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:20.260643 9169 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:20.214840 9138 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.215663 9068 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.152328 9196 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.186975 8117 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:20.063490 9186 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:20.258239 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:20.199090 9190 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260616 19:58:19.787362 9209 consensus_queue.cc:1175] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6827184e3cef42c2ada15b2d31afee4b" responder_term: 4 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.2-3.2]" } } last_received_current_leader { term: 0 index: 0 } }
I20260616 19:58:20.248512 9107 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:20.479597 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:20.480698 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.298910 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:20.319463 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 236883us
I20260616 19:58:20.485872 9004 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:20.487623 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 242531us
W20260616 19:58:20.487656 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 292759us
I20260616 19:58:20.489629 8084 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:20.347808 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:20.485921 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 366129us
I20260616 19:58:20.244506 9041 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.512501 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.487275 9198 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:20.516218 8009 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:20.260833 9195 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.521713 8084 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.324602 9133 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.396131 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.408710 9064 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.531351 7935 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:20.522681 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.540805 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:20.512977 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 364244us
I20260616 19:58:20.531381 8009 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:20.539629 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:20.541527 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:20.546175 8112 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:20.543224 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:20.544126 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 351043us
I20260616 19:58:20.544926 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:20.548136 8116 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:20.534274 9182 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:20.522738 9004 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260616 19:58:20.556129 8115 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:20.556576 7935 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.558933 8009 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:20.620659 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.537879 9039 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:20.680874 8116 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:20.527446 9198 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:20.687415 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 118056us
W20260616 19:58:20.681339 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 132101us
I20260616 19:58:20.536453 9230 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:20.545809 9195 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:20.545784 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.541863 9227 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
W20260616 19:58:20.554157 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.532370 9219 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260616 19:58:20.687727 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 131200us
I20260616 19:58:20.722807 9187 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:20.689034 8084 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:20.733700 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 44698us
I20260616 19:58:20.734601 8084 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:20.685161 8009 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:20.551987 9133 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election won for term 2
W20260616 19:58:20.680213 9218 log.cc:927] Time spent T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.079s user 0.000s sys 0.001s
I20260616 19:58:20.694702 9228 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:20.550230 9003 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.141s
I20260616 19:58:20.685317 9068 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:20.680258 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:20.696233 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.554646 9146 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.142s
W20260616 19:58:20.737046 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 51994us
I20260616 19:58:20.739601 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:20.748075 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:20.791417 8040 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:20.752087 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:20.794592 8036 raft_consensus.cc:2393] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:20.691890 9120 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260616 19:58:20.691759 9080 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260616 19:58:20.790619 8039 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:20.788108 9187 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:20.778436 9036 consensus_meta.cc:287] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.603s user 0.011s sys 0.000s
I20260616 19:58:20.775611 9138 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:20.694002 9230 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.697168 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:20.805349 9187 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.805418 8113 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:20.827875 7964 raft_consensus.cc:2393] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:20.774174 9183 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:20.842978 9183 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:20.843438 9183 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:20.844177 9230 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:20.805791 9220 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
W20260616 19:58:20.773619 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 43767.2us
I20260616 19:58:20.849473 9198 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.850050 7935 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:20.852911 9215 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.945s user 0.002s sys 0.022s
I20260616 19:58:20.697561 9222 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.110s
I20260616 19:58:20.765334 9109 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:20.780881 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:20.856144 8044 raft_consensus.cc:1240] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:20.808169 9185 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:20.788524 9195 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:20.837075 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.867918 8110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53916 (request call id 595) took 1108 ms. Trace:
I20260616 19:58:20.868537 8110 rpcz_store.cc:276] 0616 19:58:19.759410 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.760294 (+ 884us) service_pool.cc:224] Handling call
0616 19:58:19.761007 (+ 713us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:19.761334 (+ 327us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:19.761358 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:19.761802 (+ 444us) log.cc:844] Serialized 26 byte log entry
0616 19:58:19.803787 (+ 41985us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:19.803835 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:19.803903 (+ 68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:20.852310 (+1048407us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:20.859674 (+ 7364us) raft_consensus.cc:1685] finished
0616 19:58:20.859703 (+ 29us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:20.867841 (+ 8138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":720,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":33922304,"thread_start_us":41563,"threads_started":1,"wal-append.queue_time_us":62827}
I20260616 19:58:20.842198 9187 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:20.789705 9125 consensus_meta.cc:287] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 1.008s user 0.009s sys 0.003s
I20260616 19:58:20.771852 8112 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
W20260616 19:58:20.861055 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:20.787655 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.839347 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.875078 7935 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.828724 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
W20260616 19:58:20.886811 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.851994 9043 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:20.857499 9228 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:20.918936 9186 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:58:20.923601 9233 log.cc:927] Time spent T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.232s user 0.001s sys 0.000s
I20260616 19:58:20.924934 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 876) took 1079 ms. Trace:
I20260616 19:58:20.925308 7969 rpcz_store.cc:276] 0616 19:58:19.846012 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.846118 (+ 106us) service_pool.cc:224] Handling call
0616 19:58:19.846349 (+ 231us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:19.885247 (+ 38898us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:19.885292 (+ 45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:20.082889 (+197597us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b2c000328a8. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86dbdec71 00007fe86dbee5d2 00007fe86f320de1 00007fe86f2e581a 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22
0616 19:58:20.191700 (+108811us) log.cc:844] Serialized 26 byte log entry
0616 19:58:20.510739 (+319039us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:20.510814 (+ 75us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:20.510909 (+ 95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:20.922957 (+412048us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:20.924710 (+ 1753us) raft_consensus.cc:1685] finished
0616 19:58:20.924738 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:20.924876 (+ 138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23690,"raft.queue_time_us":56513,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":38839,"spinlock_wait_cycles":7533568,"thread_start_us":237718,"threads_started":1,"wal-append.queue_time_us":290024}
I20260616 19:58:20.927166 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.928145 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.836627 9107 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.881845 8040 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:20.930461 7969 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:20.935478 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.936820 8927 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:20.937132 8116 raft_consensus.cc:1240] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:20.937294 8927 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.939733 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:20.859501 9138 raft_consensus.cc:697] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:20.941830 8116 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:20.943266 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:20.942014 8927 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:20.944362 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:20.945189 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.946779 8116 raft_consensus.cc:1240] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:20.947324 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.947686 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:20.951655 8118 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:20.929503 9219 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:20.868636 9146 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260616 19:58:20.953052 8112 raft_consensus.cc:1275] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:20.952972 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:20.862850 8111 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:20.881783 9195 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.963910 8927 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:58:20.956518 9141 consensus_meta.cc:287] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 1.160s user 0.009s sys 0.003s
I20260616 19:58:20.964455 8927 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.948920 9196 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:20.967484 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:20.973287 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:20.968227 9200 consensus_meta.cc:287] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 1.150s user 0.012s sys 0.000s
I20260616 19:58:20.981577 8117 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:20.983379 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:20.982200 8927 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:20.943432 8784 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.969908 8113 pending_rounds.cc:85] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 2
I20260616 19:58:20.871068 9185 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:20.987308 8084 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:20.989009 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:20.977499 9145 consensus_meta.cc:287] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.747s user 0.011s sys 0.000s
I20260616 19:58:20.982986 9109 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.008860 9109 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.009681 9185 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.990089 9068 raft_consensus.cc:697] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:21.011044 9109 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.010998 9068 consensus_queue.cc:237] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.015008 9183 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:21.015717 9088 consensus_meta.cc:287] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 1.146s user 0.005s sys 0.005s
I20260616 19:58:20.970602 9141 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.016094 9088 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.017804 9141 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.989961 8934 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:20.989140 9138 consensus_queue.cc:237] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.035395 9088 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.035667 9143 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:20.989391 9200 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.021279 9018 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.037734 9200 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.023744 9182 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:21.002140 9145 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.025993 9183 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.029325 9241 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:21.033643 8934 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.042846 9145 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.041833 8934 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.051616 9195 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.052906 9133 raft_consensus.cc:697] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:21.053849 9133 consensus_queue.cc:237] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:21.054616 9226 log.cc:927] Time spent T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.355s user 0.001s sys 0.001s
I20260616 19:58:21.056645 9129 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.064817 9227 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:58:21.090345 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 175408us
W20260616 19:58:21.092440 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:21.093520 7969 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:21.094517 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:21.101212 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:21.102761 8115 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:21.027873 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.105746 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.107441 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.108213 8043 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 4. Ops: []
I20260616 19:58:20.989915 8113 pending_rounds.cc:107] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260616 19:58:21.108798 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.109035 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.109112 8113 raft_consensus.cc:2889] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:21.110406 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.101212 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.108287 8110 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:21.125424 9221 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:21.119886 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 573) took 1068 ms. Trace:
I20260616 19:58:21.130316 7968 rpcz_store.cc:276] 0616 19:58:20.050922 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.093322 (+1042400us) service_pool.cc:224] Handling call
0616 19:58:21.119788 (+ 26466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.131335 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.132397 9145 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:21.132895 9018 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:21.133710 9018 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:58:21.134055 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 200378us
W20260616 19:58:21.138330 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 150995us
I20260616 19:58:21.140748 8084 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.132452 8045 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:21.142071 9098 consensus_queue.cc:1059] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:21.143159 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.144023 9145 raft_consensus.cc:3055] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:21.144578 9145 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:21.143311 9183 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:21.146972 9183 raft_consensus.cc:697] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:21.148794 9183 consensus_queue.cc:237] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.153389 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:21.119995 7962 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:21.157124 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.158061 9133 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:21.157888 9227 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.173341 8042 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:21.177919 9145 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:21.172413 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 588) took 1107 ms. Trace:
I20260616 19:58:21.180102 7968 rpcz_store.cc:276] 0616 19:58:20.063863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.156637 (+1092774us) service_pool.cc:224] Handling call
0616 19:58:21.171614 (+ 14977us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.181592 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.182411 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 596) took 1115 ms. Trace:
I20260616 19:58:21.182754 7968 rpcz_store.cc:276] 0616 19:58:20.067001 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.181097 (+1114096us) service_pool.cc:224] Handling call
0616 19:58:21.182342 (+ 1245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.170679 8008 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:21.186136 8117 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:21.187713 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.135645 9018 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.171270 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 587) took 1105 ms. Trace:
I20260616 19:58:21.197912 7962 rpcz_store.cc:276] 0616 19:58:20.063529 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.100497 (+1036968us) service_pool.cc:224] Handling call
0616 19:58:21.168833 (+ 68336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.180127 9145 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:21.202121 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 875) took 1326 ms. Trace:
I20260616 19:58:21.213086 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 886) took 1363 ms. Trace:
I20260616 19:58:21.213601 7965 rpcz_store.cc:276] 0616 19:58:19.861677 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.992832 (+131155us) service_pool.cc:224] Handling call
0616 19:58:19.993091 (+ 259us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:20.528576 (+535485us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:20.528622 (+ 46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:58:20.528683 (+ 61us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:58:20.528735 (+ 52us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:21.212494 (+683759us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54003bced4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:21.212565 (+ 71us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:21.212828 (+ 263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1230866176}
I20260616 19:58:21.214871 9138 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.213485 8043 rpcz_store.cc:276] 0616 19:58:19.880246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.880409 (+ 163us) service_pool.cc:224] Handling call
0616 19:58:19.880749 (+ 340us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:21.108915 (+1228166us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:21.201730 (+ 92815us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540023d2d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:21.202009 (+ 279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821113088}
I20260616 19:58:21.214529 7968 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:21.206897 9133 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.217449 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 870) took 1377 ms. Trace:
I20260616 19:58:21.217707 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.218789 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 575) took 1167 ms. Trace:
I20260616 19:58:21.219230 7968 rpcz_store.cc:276] 0616 19:58:20.051341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.217407 (+1166066us) service_pool.cc:224] Handling call
0616 19:58:21.218728 (+ 1321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.221000 9133 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.201419 8112 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:21.216023 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.208253 9250 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:21.221082 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:21.221748 7970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 594) took 1155 ms (client timeout 1533 ms). Trace:
W20260616 19:58:21.224057 7967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 882) took 1376 ms (client timeout 1519 ms). Trace:
I20260616 19:58:21.226794 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.205837 9195 consensus_queue.cc:1059] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260616 19:58:21.217864 7966 rpcz_store.cc:276] 0616 19:58:19.839763 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.841772 (+ 2009us) service_pool.cc:224] Handling call
0616 19:58:19.842048 (+ 276us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:20.064658 (+222610us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:20.064689 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:20.065121 (+ 432us) log.cc:844] Serialized 26 byte log entry
0616 19:58:20.308595 (+243474us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:20.308658 (+ 63us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:21.053980 (+745322us) spinlock_profiling.cc:243] Waited 56.7 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:21.216829 (+162849us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b540016bf54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:21.216904 (+ 75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:21.217006 (+ 102us) raft_consensus.cc:1685] finished
0616 19:58:21.217020 (+ 14us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:21.217242 (+ 222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78168,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":45650,"spinlock_wait_cycles":641680512,"thread_start_us":183882,"threads_started":1,"wal-append.queue_time_us":429358}
I20260616 19:58:21.245366 9018 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:21.245991 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.246932 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 595) took 1180 ms. Trace:
I20260616 19:58:21.247279 7966 rpcz_store.cc:276] 0616 19:58:20.066599 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.245629 (+1179030us) service_pool.cc:224] Handling call
0616 19:58:21.246849 (+ 1220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:58:21.237115 7970 rpcz_store.cc:269] 0616 19:58:20.066367 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:20.162717 (+ 96350us) service_pool.cc:224] Handling call
0616 19:58:21.221284 (+1058567us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b540035fcd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:21.221502 (+ 218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88777472}
I20260616 19:58:21.260303 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:21.237728 7967 rpcz_store.cc:269] 0616 19:58:19.847841 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.848330 (+ 489us) service_pool.cc:224] Handling call
0616 19:58:21.223622 (+1375292us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54003f8254. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:21.223866 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844248448}
W20260616 19:58:21.266685 8011 leader_election.cc:336] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.625s (SENT)
I20260616 19:58:21.267450 8011 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:21.268101 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.269295 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 578) took 1217 ms. Trace:
I20260616 19:58:21.269703 7962 rpcz_store.cc:276] 0616 19:58:20.051958 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.260222 (+1208264us) service_pool.cc:224] Handling call
0616 19:58:21.269215 (+ 8993us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.270892 9068 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:21.272405 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 599) took 1204 ms. Trace:
I20260616 19:58:21.272807 7965 rpcz_store.cc:276] 0616 19:58:20.067849 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.215618 (+1147769us) service_pool.cc:224] Handling call
0616 19:58:21.272328 (+ 56710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.274787 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 591) took 1209 ms. Trace:
W20260616 19:58:21.275157 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 77902.2us
I20260616 19:58:21.276268 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.277415 9133 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:21.278609 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 580) took 1226 ms. Trace:
I20260616 19:58:21.278949 7966 rpcz_store.cc:276] 0616 19:58:20.052472 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.247953 (+1195481us) service_pool.cc:224] Handling call
0616 19:58:21.278557 (+ 30604us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.275163 7965 rpcz_store.cc:276] 0616 19:58:20.065535 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.273526 (+1207991us) service_pool.cc:224] Handling call
0616 19:58:21.273961 (+ 435us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:21.274362 (+ 401us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:21.274390 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:58:21.274462 (+ 72us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:21.274492 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:21.274555 (+ 63us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:21.274690 (+ 135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.271864 7962 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260616 19:58:21.215847 9138 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.286820 7966 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260616 19:58:21.280472 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 75216.8us
I20260616 19:58:21.297839 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 597) took 1224 ms. Trace:
I20260616 19:58:21.298312 7966 rpcz_store.cc:276] 0616 19:58:20.067250 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.279562 (+1212312us) service_pool.cc:224] Handling call
0616 19:58:21.279949 (+ 387us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:21.291098 (+ 11149us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:21.291376 (+ 278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.303205 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 880) took 1455 ms. Trace:
I20260616 19:58:21.303716 7963 rpcz_store.cc:276] 0616 19:58:19.847265 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.869047 (+ 21782us) service_pool.cc:224] Handling call
0616 19:58:21.301406 (+1432359us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400259254. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:21.303004 (+ 1598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448342912}
I20260616 19:58:21.313063 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 577) took 1261 ms. Trace:
I20260616 19:58:21.313532 7969 rpcz_store.cc:276] 0616 19:58:20.051761 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:20.944970 (+893209us) service_pool.cc:224] Handling call
0616 19:58:21.312799 (+367829us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003f7d54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:21.312975 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338409856}
I20260616 19:58:21.314239 9106 raft_consensus.cc:3055] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:21.314752 9106 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:58:21.316215 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 176657us
I20260616 19:58:21.290409 7962 pending_rounds.cc:85] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 2
I20260616 19:58:21.315948 9106 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.316634 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 589) took 1252 ms. Trace:
I20260616 19:58:21.338595 7964 rpcz_store.cc:276] 0616 19:58:20.064169 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:20.691588 (+627419us) service_pool.cc:224] Handling call
0616 19:58:21.316216 (+624628us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b54003007d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:21.316433 (+ 217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70195968}
I20260616 19:58:21.319155 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.339732 9068 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.340538 9068 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.340973 8044 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:21.320581 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.320590 8036 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:21.342797 9068 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.302103 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.340904 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.329596 8043 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:21.330298 8040 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:21.321807 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.331151 9049 consensus_queue.cc:1059] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260616 19:58:21.348974 9049 raft_consensus.cc:3055] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:21.349365 9049 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:21.349537 9034 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.350427 9049 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.350805 9034 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.330904 8784 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:21.353298 9244 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:21.354827 8009 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.355814 9068 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:21.356357 9068 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.356956 9068 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:21.358609 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 23) took 1604 ms. Trace:
I20260616 19:58:21.359031 7864 rpcz_store.cc:276] 0616 19:58:19.754581 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:19.757800 (+ 3219us) service_pool.cc:224] Handling call
0616 19:58:19.972186 (+214386us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:21.358410 (+1386224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184858,"reported_tablets":1,"spinlock_wait_cycles":157982464}
I20260616 19:58:21.329450 7934 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:21.336473 8042 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:21.351902 9049 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.357138 8036 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:21.323897 7962 pending_rounds.cc:107] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260616 19:58:21.331344 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 119) took 4110 ms. Trace:
I20260616 19:58:21.338152 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.335561 9138 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.337680 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.374037 7961 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:21.374487 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 601) took 1291 ms. Trace:
I20260616 19:58:21.374727 8913 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.374928 7969 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:21.375356 8913 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:21.379679 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:21.371100 7968 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:21.320041 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.334734 9133 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.383666 8913 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.383934 9133 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.386116 9133 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:21.386775 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 57484.5us
I20260616 19:58:21.386945 9195 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:21.387383 9195 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.387753 9195 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.388306 9221 consensus_queue.cc:1059] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:21.389590 9221 consensus_queue.cc:1048] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:21.390419 9035 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:21.391148 9221 raft_consensus.cc:3055] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:21.391520 9221 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:21.374760 7961 rpcz_store.cc:276] 0616 19:58:20.083428 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.250045 (+1166617us) service_pool.cc:224] Handling call
0616 19:58:21.321566 (+ 71521us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:58:21.374434 (+ 52868us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8783616}
I20260616 19:58:21.300547 9250 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.399108 9250 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.402067 9143 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.402881 8084 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.402773 9143 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.404387 7962 raft_consensus.cc:2889] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:21.380867 8041 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:21.383225 9138 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.391449 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:21.417191 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:21.433372 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 590) took 1369 ms. Trace:
I20260616 19:58:21.434003 9143 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.331274 9106 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:21.474855 9256 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.057s user 0.002s sys 0.000s
I20260616 19:58:21.381974 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 585) took 1328 ms. Trace:
I20260616 19:58:21.440431 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:21.445574 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.446310 9250 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:21.446365 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:21.452981 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 50268.5us
I20260616 19:58:21.457872 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.459235 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.489360 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.459615 9138 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.490217 7967 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:21.460307 8043 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:21.462790 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.463613 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.337849 8934 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.466590 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.467386 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.454507 9034 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.470686 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.501466 8117 raft_consensus.cc:2393] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:21.471748 9045 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.472357 9033 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:21.454427 7962 rpcz_store.cc:276] 0616 19:58:20.065271 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.270598 (+1205327us) service_pool.cc:224] Handling call
0616 19:58:21.271184 (+ 586us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:21.433062 (+161878us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:21.433287 (+ 225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.510443 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.511274 8043 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:21.476176 7968 rpcz_store.cc:276] 0616 19:58:20.053518 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.220046 (+1166528us) service_pool.cc:224] Handling call
0616 19:58:21.381907 (+161861us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:58:21.512360 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.484021 7969 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:21.486529 8009 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.486822 9236 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:21.487823 9250 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.488516 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.488785 8036 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.489239 8042 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.491345 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.455381 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:21.504446 8037 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:21.509171 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.466642 7864 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:21.509747 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.460326 9221 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.513128 9045 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.537535 8934 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.534776 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.518286 7964 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
W20260616 19:58:21.571992 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 85612.5us
I20260616 19:58:21.570729 8043 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:21.521947 9145 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:21.538245 9045 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.579434 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.579435 8118 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:21.574447 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.580535 9240 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:21.510505 9033 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:21.545437 8011 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.884s (SENT) [suppressed 22 similar messages]
I20260616 19:58:21.581954 9236 raft_consensus.cc:697] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:21.593310 9250 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.617893 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.594233 9138 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.528810 8041 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:21.618625 9045 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:21.619015 9045 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.619342 9045 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.619463 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.371500 7866 rpcz_store.cc:276] 0616 19:58:17.228957 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:17.232450 (+ 3493us) service_pool.cc:224] Handling call
0616 19:58:17.577610 (+345160us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:21.331095 (+3753485us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:19.499439 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:19.539867 (+ 40428us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:19.539899 (+ 32us) write_op.cc:435] Acquired schema lock
0616 19:58:19.539922 (+ 23us) tablet.cc:662] Decoding operations
0616 19:58:19.563056 (+ 23134us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:19.563150 (+ 94us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:19.563195 (+ 45us) tablet.cc:685] Acquiring locks for 7 operations
0616 19:58:19.563941 (+ 746us) tablet.cc:701] Row locks acquired
0616 19:58:19.563968 (+ 27us) write_op.cc:260] PREPARE: finished
0616 19:58:19.564140 (+ 172us) write_op.cc:270] Start()
0616 19:58:19.564294 (+ 154us) write_op.cc:276] Timestamp: P: 1781639899564092 usec, L: 0
0616 19:58:19.564328 (+ 34us) op_driver.cc:348] REPLICATION: starting
0616 19:58:19.600347 (+ 36019us) log.cc:844] Serialized 2927 byte log entry
0616 19:58:19.786125 (+185778us) op_driver.cc:464] REPLICATION: finished
0616 19:58:19.909292 (+123167us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e1208 00007fe872d1d401 00007fe872d1cbd8 00007fe872d21871 00007fe872d21829 00007fe872d217b9 00007fe872d21781 00007fe872d209ad 0000000000389354 0000000000389199
0616 19:58:20.021693 (+112401us) write_op.cc:301] APPLY: starting
0616 19:58:20.027445 (+ 5752us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:20.119405 (+ 91960us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:20.142208 (+ 22803us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:20.208693 (+ 66485us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:20.208927 (+ 234us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0616 19:58:20.208981 (+ 54us) write_op.cc:312] APPLY: finished
0616 19:58:20.209273 (+ 292us) log.cc:844] Serialized 58 byte log entry
0616 19:58:20.209457 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:20.209588 (+ 131us) write_op.cc:454] Released schema lock
0616 19:58:20.209643 (+ 55us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:20.210041 (+ 398us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:20.210088 (+ 47us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":679201536,"child_traces":[["op",{"apply.queue_time_us":194459,"apply.run_cpu_time_us":4657,"apply.run_wall_time_us":210931,"num_ops":7,"prepare.queue_time_us":22689,"prepare.run_cpu_time_us":60121,"prepare.run_wall_time_us":465724,"raft.queue_time_us":30497,"raft.run_cpu_time_us":15797,"raft.run_wall_time_us":83172,"replication_time_us":191390,"spinlock_wait_cycles":28769152,"thread_start_us":123751,"threads_started":4,"wal-append.queue_time_us":80166,"wal-append.run_cpu_time_us":6412,"wal-append.run_wall_time_us":1531750}]]}
I20260616 19:58:21.621025 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.610544 8044 raft_consensus.cc:2393] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:58:21.609786 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 108320us
I20260616 19:58:21.617522 9236 consensus_queue.cc:237] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:21.618037 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 138190us
I20260616 19:58:21.594875 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.619287 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.595446 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.642606 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:21.639063 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 105612us
I20260616 19:58:21.649647 8042 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:21.627743 9045 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.651393 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.676896 8036 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:21.652678 8037 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:21.656109 7970 raft_consensus.cc:2410] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260616 19:58:21.656384 9258 log.cc:927] Time spent T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260616 19:58:21.660965 8934 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:21.663421 9143 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:21.689993 9143 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.691082 9045 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.664330 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.692122 7962 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:21.665428 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.665428 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.693126 7967 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:21.693364 8117 raft_consensus.cc:2393] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:21.607177 9145 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.670157 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.671896 8045 raft_consensus.cc:1240] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:21.610091 9240 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:21.684851 8119 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:21.694248 8044 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260616 19:58:21.696533 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:21.708312 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.694607 8043 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:21.701349 9143 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.682156 9221 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.704808 8084 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.709259 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:21.704766 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.721702 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.722110 8084 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.700951 9145 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:21.720547 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 271690us
I20260616 19:58:21.700141 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:21.719281 9049 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:21.733469 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.716831 9240 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.740227 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.743160 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.744390 7963 raft_consensus.cc:2393] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:21.741089 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 894) took 1225 ms. Trace:
I20260616 19:58:21.745174 7970 rpcz_store.cc:276] 0616 19:58:20.516038 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.249405 (+733367us) service_pool.cc:224] Handling call
0616 19:58:21.299939 (+ 50534us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:58:21.735437 (+435498us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b540004c9d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:21.741019 (+ 5582us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689195648}
I20260616 19:58:21.748124 8112 raft_consensus.cc:2410] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:21.733820 9045 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:21.747816 7969 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:21.742507 8084 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.755573 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.754094 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.769182 8039 raft_consensus.cc:1240] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:21.762988 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:21.773375 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:21.748175 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.774983 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.778896 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.750959 9045 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.790541 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.792774 8045 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:21.792876 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.798853 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.800458 9035 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260616 19:58:21.803225 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.803440 9126 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:21.803790 9126 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:21.803953 8045 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:21.804972 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:21.804737 9126 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.805554 8045 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.806011 9126 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.807610 9097 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:21.808058 9097 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:21.808454 9097 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.810935 9195 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.812314 9018 consensus_queue.cc:1059] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:21.787981 9045 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:21.795388 8111 raft_consensus.cc:2410] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:21.794468 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:21.813228 9035 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:21.818660 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.820967 8084 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.769683 9068 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.761092 9143 consensus_queue.cc:1059] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260616 19:58:21.800956 8934 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:21.813850 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.833891 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.835302 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:21.835938 8036 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:21.836843 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:58:21.839054 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 49448.3us
I20260616 19:58:21.840096 9200 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:21.840711 9200 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.842847 9200 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.843670 8946 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:21.844225 8946 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:21.845163 8946 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.848045 9033 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.856503 7964 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:21.826265 9035 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:21.862522 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.863826 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.864974 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.865546 8042 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:21.873202 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.873415 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.877225 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:58:21.887869 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 56620.3us
W20260616 19:58:21.896486 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 57222.4us
I20260616 19:58:21.898114 9195 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:21.900978 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:21.833331 8934 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.909662 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.908844 8084 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.907274 8045 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
W20260616 19:58:21.911306 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 148443us
I20260616 19:58:21.913123 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.913817 7969 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:21.916077 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:21.852136 9097 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:21.918123 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 78468.2us
I20260616 19:58:21.917547 8038 raft_consensus.cc:2368] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:21.926421 8045 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260616 19:58:21.930820 8011 leader_election.cc:336] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 2.148s (SENT)
I20260616 19:58:21.931298 8011 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:21.935174 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:21.935823 8040 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:58:21.914988 8934 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:21.943375 8084 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.944214 8913 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:21.944561 8913 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.944878 8913 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:21.920954 9033 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:21.945930 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:21.948311 9129 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.948764 9097 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.948916 9129 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.950347 9230 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:21.950740 9230 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:21.951107 9230 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:21.951292 9129 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.951736 9198 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:21.954319 9240 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.957027 9068 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:21.959218 8913 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.951052 7964 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:21.960865 8041 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:21.938923 9183 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:21.965155 9183 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.966913 9045 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:21.967417 9183 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.969214 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.970130 9045 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.970949 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.971145 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.971748 7967 raft_consensus.cc:2410] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:21.978070 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:21.979220 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:21.994638 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:22.009392 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 46945.7us
W20260616 19:58:22.034556 9275 log.cc:927] Time spent T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.066s user 0.001s sys 0.000s
I20260616 19:58:22.038647 8913 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:21.980628 9033 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:22.041682 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.030800 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.049686 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.047746 8085 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.052052 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.053082 7969 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:22.053748 7965 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
W20260616 19:58:22.060186 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 148872us
I20260616 19:58:22.046429 9240 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:22.066807 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.069814 9033 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.073571 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.074656 7969 raft_consensus.cc:2393] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:22.063356 9097 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:22.089107 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:22.089879 8113 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:22.090150 9106 consensus_meta.cc:287] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.615s user 0.011s sys 0.000s
I20260616 19:58:22.073896 7964 raft_consensus.cc:2393] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:22.100965 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.103300 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.106086 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:22.106109 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 64162.9us
I20260616 19:58:22.110814 7961 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:22.092989 9230 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.116468 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.113947 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.119036 8036 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:22.119982 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.121730 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:22.103842 9273 log.cc:927] Time spent T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.067s user 0.001s sys 0.000s
I20260616 19:58:22.105037 9097 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:22.161275 8085 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.163008 9145 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.166528 7961 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:22.168365 7965 raft_consensus.cc:2410] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:22.169642 7965 raft_consensus.cc:1275] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:22.170769 7965 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:22.171097 7965 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:22.171993 7965 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.173493 7965 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:22.180655 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 110762us
W20260616 19:58:22.173483 9266 log.cc:927] Time spent T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.349s user 0.001s sys 0.005s
I20260616 19:58:22.180720 7962 raft_consensus.cc:2410] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260616 19:58:22.214380 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 53271.9us
I20260616 19:58:22.219192 9088 raft_consensus.cc:1064] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:22.219321 9097 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.221650 9230 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:22.221671 9088 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:22.224721 8039 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260616 19:58:22.225689 9143 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:22.226516 9143 raft_consensus.cc:3055] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:22.226742 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:22.227485 8039 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:22.226830 9143 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:22.228711 9143 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.229960 9143 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:22.231402 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:22.231875 8118 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:22.232141 8115 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:22.232272 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:22.230091 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.233969 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:22.235363 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:22.236112 8036 raft_consensus.cc:2410] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:22.236284 9171 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:22.236665 9171 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:22.237025 9171 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.236941 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:22.237551 8039 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.237977 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:22.239698 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:22.240475 8110 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:22.251731 8039 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.252776 7965 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260616 19:58:22.253193 7965 pending_rounds.cc:85] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:22.253460 7965 pending_rounds.cc:107] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260616 19:58:22.253742 7965 raft_consensus.cc:2889] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:22.246209 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:22.258169 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.262326 8038 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.275259 8934 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.294943 8934 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:22.296269 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.296190 8820 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260616 19:58:22.306001 9221 consensus_meta.cc:287] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.582s user 0.006s sys 0.003s
I20260616 19:58:22.307058 7963 raft_consensus.cc:2410] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:22.308749 9145 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:22.309271 7961 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260616 19:58:22.309743 8113 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:22.310628 8114 raft_consensus.cc:1240] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:22.309765 7961 pending_rounds.cc:85] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:22.313267 7961 pending_rounds.cc:107] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260616 19:58:22.313570 7961 raft_consensus.cc:2889] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260616 19:58:22.320238 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:22.340965 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 50686.2us
I20260616 19:58:22.344116 9200 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:22.352362 7934 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:22.341089 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.364140 8037 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:22.352835 8041 raft_consensus.cc:1240] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:22.368904 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:22.373142 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:22.378630 8040 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:22.287236 9129 consensus_queue.cc:1059] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260616 19:58:22.362924 9282 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260616 19:58:22.382405 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 99013us
W20260616 19:58:22.385913 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.387147 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:22.388120 8041 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:22.395601 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:22.398535 8040 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:22.391192 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.389426 9282 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:22.409168 8038 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.411187 9045 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:22.414153 9035 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.418879 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.482288 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.457487 9282 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:22.490572 8110 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:22.491775 9221 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:22.492317 9264 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.697s user 0.004s sys 0.000s
I20260616 19:58:22.493510 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 898) took 1976 ms. Trace:
I20260616 19:58:22.493592 7935 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.493904 7966 rpcz_store.cc:276] 0616 19:58:20.517200 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.306093 (+788893us) service_pool.cc:224] Handling call
0616 19:58:21.306735 (+ 642us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:21.307392 (+ 657us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:21.307445 (+ 53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:21.352598 (+ 45153us) log.cc:844] Serialized 26 byte log entry
0616 19:58:21.448550 (+ 95952us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:21.448616 (+ 66us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:21.448741 (+ 125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:22.468394 (+1019653us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:22.493216 (+ 24822us) raft_consensus.cc:1685] finished
0616 19:58:22.493237 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:22.493452 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1318,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"spinlock_wait_cycles":104564992,"thread_start_us":95307,"threads_started":1,"wal-append.queue_time_us":400622}
W20260616 19:58:22.495253 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.496269 9221 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260616 19:58:22.496271 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 101342us
I20260616 19:58:22.497140 8009 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:22.498396 8011 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.499630 8009 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:22.500399 9240 raft_consensus.cc:2764] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260616 19:58:22.501178 9240 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:22.502946 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.503455 9240 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:22.503854 9200 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:22.504060 9240 raft_consensus.cc:697] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:22.505304 8011 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.505133 9240 consensus_queue.cc:237] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:22.508620 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.512583 8011 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.513383 9236 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:22.513938 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.507493 9171 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.514808 9129 consensus_queue.cc:1059] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:22.516853 9171 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:22.517518 9272 log.cc:927] Time spent T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.468s user 0.000s sys 0.006s
I20260616 19:58:22.517457 8112 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:22.520824 9236 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:22.489528 9035 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:22.521389 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.518112 9049 raft_consensus.cc:3055] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:22.528144 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.528913 7970 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:22.511428 9200 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:22.526566 9049 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:22.521049 9129 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260616 19:58:22.534710 9049 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.535954 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.536213 9049 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:22.538216 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.538632 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.540411 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.540745 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.541019 7963 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
W20260616 19:58:22.541303 9126 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260616 19:58:22.542215 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.543001 7966 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
W20260616 19:58:22.531944 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 127414us
I20260616 19:58:22.518954 9221 consensus_queue.cc:1059] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260616 19:58:22.575850 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 54597.2us
W20260616 19:58:22.583372 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.587972 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:22.583837 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.588874 8114 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:22.590662 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:22.596221 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.596808 7961 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.590426 7935 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.586395 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:22.598584 8116 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:22.595839 8113 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260616 19:58:22.603346 8117 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260616 19:58:22.540439 9300 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:22.590598 9182 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:22.646126 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 54738.1us
I20260616 19:58:22.575564 9195 consensus_queue.cc:1059] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260616 19:58:22.635360 8119 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:22.691402 8112 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:22.693662 7970 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:22.699865 8011 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:22.701253 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 65851.3us
I20260616 19:58:22.704437 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.705125 7970 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.706962 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:22.693956 8040 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
W20260616 19:58:22.696417 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 112911us
I20260616 19:58:22.708418 7935 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:22.712392 8116 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:22.714162 7961 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.698753 9221 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:22.710521 9200 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:22.727157 7970 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.729379 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:22.733096 8111 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260616 19:58:22.738358 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 48723.4us
I20260616 19:58:22.741261 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.747080 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.748673 8011 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.750221 7961 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.757028 9200 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:22.743456 9300 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:22.798185 9126 raft_consensus.cc:3055] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:22.758810 9295 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:22.798731 9126 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:22.799088 9295 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.801254 9295 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:22.803975 8038 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260616 19:58:22.779278 9236 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:22.804580 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:22.805222 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:22.805902 8042 raft_consensus.cc:2368] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
I20260616 19:58:22.806231 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:22.759822 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 51462us
I20260616 19:58:22.780084 9133 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:22.816880 7967 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260616 19:58:22.817389 7967 pending_rounds.cc:85] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:22.817612 7967 pending_rounds.cc:107] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260616 19:58:22.817545 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:22.818077 7967 raft_consensus.cc:2889] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:22.818308 8039 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.799590 9126 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.782874 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.783495 7961 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.824602 9129 raft_consensus.cc:3055] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:22.722430 9182 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.825024 9129 raft_consensus.cc:740] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:22.825362 8011 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.826197 9236 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:22.826056 9129 consensus_queue.cc:260] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.826853 9236 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:22.827126 9182 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:22.827351 9129 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:22.820041 9126 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:22.828111 8115 raft_consensus.cc:1275] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:22.831775 7935 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.833065 7935 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.834018 9301 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:22.836680 9221 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:22.839294 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:22.840221 7970 raft_consensus.cc:2368] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
W20260616 19:58:22.843830 9302 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.206s user 0.000s sys 0.000s
W20260616 19:58:22.845103 9298 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.141s user 0.000s sys 0.000s
I20260616 19:58:22.837949 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.847272 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.848254 9126 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:22.849053 9035 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:22.798321 9300 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:22.849870 9300 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:22.805259 8044 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.852931 7934 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:22.808398 8040 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.854934 8085 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:22.779706 8009 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:22.796049 9282 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:22.856482 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 76854.2us
I20260616 19:58:22.857800 8009 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:22.858563 8044 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.847697 9018 raft_consensus.cc:3055] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:22.859225 9133 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:22.859485 9018 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:22.859699 9133 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:22.860096 9133 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:22.860607 9018 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.862643 9018 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:22.864305 9236 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:22.865262 9221 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:22.865669 9236 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:22.866281 9236 raft_consensus.cc:697] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:22.867230 9236 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.868305 9200 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:22.868266 9282 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:22.871712 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:22.872288 8115 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:22.875120 9300 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.875473 9033 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:22.876129 9033 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.877466 9295 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:22.877856 9295 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:22.878167 9295 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:22.878527 9033 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:22.882305 8040 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.882475 9098 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:22.883140 9098 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:22.884482 9292 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:22.885068 9292 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:22.865739 9221 raft_consensus.cc:697] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:22.833096 8011 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:22.887329 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.848546 7935 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.887249 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.887837 8039 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.888940 8085 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.896912 8115 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:22.897953 7965 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:22.902355 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.903115 7965 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:22.908499 7934 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:22.896307 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:22.916247 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 67757us
I20260616 19:58:22.917351 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:22.922971 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:22.923888 7968 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:22.920197 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:22.925311 9045 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:22.938210 8085 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:22.938104 9182 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:22.949229 9049 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:22.956331 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.959046 9292 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:22.961138 8115 raft_consensus.cc:1240] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:22.999379 8119 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:22.935791 9195 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
W20260616 19:58:22.973363 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:22.969235 8085 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:23.000687 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 68747.3us
I20260616 19:58:23.025285 8084 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:22.988883 9294 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:22.972931 9045 raft_consensus.cc:697] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.029127 9049 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:23.032728 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:23.000578 9264 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
W20260616 19:58:23.036775 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 67663.2us
I20260616 19:58:23.043071 8084 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.026584 9097 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:23.022915 9182 raft_consensus.cc:697] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:23.048645 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 189866us
I20260616 19:58:23.026335 9033 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260616 19:58:23.036538 9292 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:23.050844 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 191108us
I20260616 19:58:23.052183 8913 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.039979 8946 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:23.038436 9195 raft_consensus.cc:697] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:23.046278 9294 raft_consensus.cc:697] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:23.059845 9306 log.cc:927] Time spent T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260616 19:58:23.060911 9182 consensus_queue.cc:237] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.050357 9034 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:23.041764 9133 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.031436 9221 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.045307 9049 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.063853 9300 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.066588 8946 raft_consensus.cc:697] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.058472 9292 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.067363 9034 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:23.080960 9294 consensus_queue.cc:237] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.082512 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:23.082906 9097 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:58:23.100749 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 183495us
I20260616 19:58:23.106405 7935 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.085633 9236 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.100055 9034 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:23.104668 9097 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.085016 9098 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:23.114078 9106 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:23.127580 9312 log.cc:927] Time spent T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260616 19:58:23.122365 9133 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.103495 9045 consensus_queue.cc:237] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.136407 9316 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:23.137404 9315 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:23.114300 9301 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.122489 8820 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:23.140017 8946 consensus_queue.cc:237] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:23.188845 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 136972us
I20260616 19:58:23.109674 9300 consensus_queue.cc:1059] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260616 19:58:23.155814 7935 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:23.186483 9293 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:23.142344 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 24) took 1698 ms. Trace:
W20260616 19:58:23.193372 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 360308us
W20260616 19:58:23.197233 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:23.157043 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.201967 8011 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:23.204670 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 152893us
I20260616 19:58:23.192957 9308 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:23.211843 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 56148us
I20260616 19:58:23.214680 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:58:23.215679 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 164736us
I20260616 19:58:23.207742 7866 catalog_manager.cc:5696] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:23.220944 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.156503 9195 consensus_queue.cc:237] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.209595 9323 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:23.200479 9293 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.203135 9315 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.213760 9133 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260616 19:58:23.233968 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:23.222002 9240 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:23.237591 8085 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.233331 7866 catalog_manager.cc:5696] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:23.242106 7935 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.228024 9323 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:23.231125 9106 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.252969 9098 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:23.253468 9317 raft_consensus.cc:3055] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:23.252880 7866 catalog_manager.cc:5696] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:23.254092 9221 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:23.257926 8082 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:58:23.267874 8913 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:23.268373 8913 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.268779 8913 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.252831 9295 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.240006 9240 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:23.272893 9240 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.274273 9295 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.276348 9098 raft_consensus.cc:697] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.277423 9098 consensus_queue.cc:237] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.249145 9323 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.282989 8085 leader_election.cc:400] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
I20260616 19:58:23.283354 8085 leader_election.cc:403] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260616 19:58:23.285852 9315 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.290419 9292 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.214650 9308 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:23.238917 9049 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.292347 9295 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
I20260616 19:58:23.292789 9293 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.238798 9280 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:23.293272 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 52199.3us
I20260616 19:58:23.285907 8011 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:23.295166 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.296188 8913 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:23.298532 9326 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:23.299820 9295 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:23.244065 9325 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260616 19:58:23.300267 9295 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.300735 9295 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:23.303720 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 61725us
I20260616 19:58:23.304647 7935 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:58:23.313696 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:23.317435 8913 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:23.316021 7866 catalog_manager.cc:5696] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
W20260616 19:58:23.326906 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 107825us
I20260616 19:58:23.327687 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:58:23.336495 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.337222 7866 catalog_manager.cc:5696] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:23.341297 9308 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.342161 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:23.342537 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 56738.6us
I20260616 19:58:23.342993 8039 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:58:23.343135 9129 consensus_meta.cc:287] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.503s user 0.006s sys 0.009s
I20260616 19:58:23.343322 8011 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.341544 7866 catalog_manager.cc:5696] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:23.198856 7864 rpcz_store.cc:276] 0616 19:58:21.444579 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:21.460753 (+ 16174us) service_pool.cc:224] Handling call
0616 19:58:23.142268 (+1681515us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:22.057000 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:22.090005 (+ 33005us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:22.090060 (+ 55us) write_op.cc:435] Acquired schema lock
0616 19:58:22.090076 (+ 16us) tablet.cc:662] Decoding operations
0616 19:58:22.090471 (+ 395us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:22.090560 (+ 89us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:22.090592 (+ 32us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:22.090922 (+ 330us) tablet.cc:701] Row locks acquired
0616 19:58:22.090943 (+ 21us) write_op.cc:260] PREPARE: finished
0616 19:58:22.091131 (+ 188us) write_op.cc:270] Start()
0616 19:58:22.091271 (+ 140us) write_op.cc:276] Timestamp: P: 1781639902091067 usec, L: 0
0616 19:58:22.091309 (+ 38us) op_driver.cc:348] REPLICATION: starting
0616 19:58:22.091993 (+ 684us) log.cc:844] Serialized 598 byte log entry
0616 19:58:22.720596 (+628603us) op_driver.cc:464] REPLICATION: finished
0616 19:58:23.052749 (+332153us) write_op.cc:301] APPLY: starting
0616 19:58:23.063753 (+ 11004us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:23.074172 (+ 10419us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:23.078467 (+ 4295us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:23.078763 (+ 296us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:23.079024 (+ 261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:23.079098 (+ 74us) write_op.cc:312] APPLY: finished
0616 19:58:23.079373 (+ 275us) log.cc:844] Serialized 22 byte log entry
0616 19:58:23.079567 (+ 194us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:23.079651 (+ 84us) write_op.cc:454] Released schema lock
0616 19:58:23.079717 (+ 66us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:23.080137 (+ 420us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:23.080197 (+ 60us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":311127,"apply.run_cpu_time_us":2536,"apply.run_wall_time_us":39614,"num_ops":1,"prepare.queue_time_us":423372,"prepare.run_cpu_time_us":26946,"prepare.run_wall_time_us":245731,"raft.queue_time_us":278121,"raft.run_cpu_time_us":36192,"raft.run_wall_time_us":205281,"replication_time_us":629113,"thread_start_us":754881,"threads_started":4,"wal-append.queue_time_us":290336}]]}
I20260616 19:58:23.346397 8084 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.272204 9317 raft_consensus.cc:740] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:23.315820 9316 raft_consensus.cc:2764] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260616 19:58:23.350163 8011 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.345831 7866 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:23.323643 9300 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:23.353330 9300 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:23.353714 9300 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:23.353801 9033 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:23.348286 9236 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:23.347903 8820 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.353717 9292 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:23.352118 9301 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.356853 9301 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.357980 9097 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:23.359045 9301 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.351750 9282 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.335616 9295 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.360030 9282 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.360721 8011 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.361723 9295 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.362457 9282 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.362948 8011 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:23.364609 9323 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.351922 9049 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.371692 7935 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.371670 8084 leader_election.cc:400] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
I20260616 19:58:23.372975 9133 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:23.373098 9049 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.373732 9236 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.374459 9323 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.375499 9049 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.375247 7866 catalog_manager.cc:5696] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:23.376187 9106 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.376780 9106 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.377187 7866 catalog_manager.cc:5696] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:23.378707 7866 catalog_manager.cc:5696] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:23.379312 9106 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.380430 9292 raft_consensus.cc:697] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.380966 9317 consensus_queue.cc:260] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.381392 9292 consensus_queue.cc:237] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.382659 9317 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:23.384502 9316 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:23.384887 9316 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:23.385252 9316 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:23.385953 9034 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.387195 8111 raft_consensus.cc:1240] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:23.390780 9282 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.391355 9282 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.391716 9106 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:23.391772 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.392621 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.392652 8119 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:23.393822 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.394531 8115 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:23.397686 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:23.398381 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 46088.2us
I20260616 19:58:23.401664 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:23.401759 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 103912us
I20260616 19:58:23.402493 8115 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:23.402705 8085 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.405100 9106 consensus_queue.cc:1059] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:23.405495 8820 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:23.406453 9106 raft_consensus.cc:3055] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:23.407089 9236 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:58:23.407110 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:23.406840 9106 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:23.408344 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.404349 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.330432 9240 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.416457 7934 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.420938 9330 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:23.422014 7935 leader_election.cc:400] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:23.422422 7935 leader_election.cc:403] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:23.420310 7934 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.428059 8082 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:58:23.432485 9301 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.433429 9049 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:23.433871 9049 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.434242 9049 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.414568 9323 consensus_queue.cc:1059] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260616 19:58:23.436849 9221 raft_consensus.cc:3055] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:23.432528 9106 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.437268 9221 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:23.437945 9106 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:23.438228 9221 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.438793 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.439779 9221 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.439713 9319 consensus_queue.cc:1059] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:23.440902 9318 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.441100 9126 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:23.441607 9126 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:23.418969 9129 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:23.442371 9129 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.442938 9129 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.442461 9126 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.445163 7935 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.446637 9320 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.417716 9097 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.449247 9282 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.424181 9280 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:23.450626 9280 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.451093 9280 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:23.454321 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:23.444000 9126 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:23.470006 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.470688 8036 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:23.472411 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.421316 9330 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.477742 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:23.477952 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 49991.1us
W20260616 19:58:23.478297 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 65399.3us
I20260616 19:58:23.479357 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.483994 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.484660 8037 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:23.486816 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:23.487787 8044 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:23.488291 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.488710 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.488880 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:23.488927 8036 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:23.489439 8045 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:23.489729 7967 raft_consensus.cc:2410] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:23.490357 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.491199 7935 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:23.493429 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:23.493991 8039 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:23.494164 8114 raft_consensus.cc:1275] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260616 19:58:23.440936 9319 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.497421 7935 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:23.449293 9034 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.499400 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:23.502271 8011 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.502835 7968 raft_consensus.cc:2410] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:23.501449 8116 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:23.503643 7969 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:23.504868 8117 raft_consensus.cc:1240] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:23.504690 8084 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.519115 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:23.528929 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.529124 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:23.529742 7968 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.494995 9033 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:23.501773 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:23.538846 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:23.539424 7965 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:23.541025 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 53505.1us
I20260616 19:58:23.541810 8011 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.543303 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.496945 9320 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:23.502065 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.495740 9195 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.556381 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.590066 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:23.591173 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49447.8us
I20260616 19:58:23.565029 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.557713 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.562738 7968 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:23.581669 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.507606 9035 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:23.594575 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:23.530397 9098 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:23.600121 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.486112 9330 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:23.601259 8011 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:23.596660 7965 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:23.588757 8038 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.538657 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:23.603006 8116 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:23.623858 8045 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:23.548017 9323 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:23.630216 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:23.577397 9320 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.602751 8113 raft_consensus.cc:2393] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:23.535383 9236 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:23.626921 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:23.537961 9315 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260616 19:58:23.512796 9319 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.638702 7965 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:23.639065 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:23.644541 7969 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.644676 7965 raft_consensus.cc:1240] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:23.601691 9035 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.629171 9323 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.586570 9300 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.626320 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.642765 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:23.632929 9033 raft_consensus.cc:697] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.597733 9195 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.640867 9133 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
W20260616 19:58:23.652063 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 60264.5us
I20260616 19:58:23.632333 9320 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:23.689373 7961 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260616 19:58:23.702276 7961 pending_rounds.cc:85] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:23.702553 7961 pending_rounds.cc:107] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260616 19:58:23.702858 7961 raft_consensus.cc:2889] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:23.699478 8085 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.650676 9129 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.652431 9327 consensus_queue.cc:1059] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.119s
I20260616 19:58:23.707113 9236 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:23.709177 7966 raft_consensus.cc:1240] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:23.710012 8011 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:23.688479 9334 log.cc:927] Time spent T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.051s user 0.001s sys 0.000s
I20260616 19:58:23.640425 9098 raft_consensus.cc:697] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.719096 9129 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.651656 9035 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.657015 9323 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.687427 9324 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:23.709770 9240 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.750250 7968 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:23.708282 9319 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.753095 7969 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
W20260616 19:58:23.764951 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 136872us
I20260616 19:58:23.769819 8084 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.719897 9045 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:23.687629 9328 consensus_queue.cc:1048] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.170s
W20260616 19:58:23.777114 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 76232.6us
I20260616 19:58:23.708669 9049 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.784978 9324 raft_consensus.cc:3055] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:23.797009 9045 raft_consensus.cc:697] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.800226 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.804756 8040 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.794157 9034 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.794343 9292 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:23.827493 8037 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:23.801090 8784 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.828847 9308 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:23.807293 9280 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.829748 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.830572 7970 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:23.831162 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.831748 9280 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.832345 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.833171 9300 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.833235 8040 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:23.799150 9348 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:23.838477 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.802712 9324 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:23.835161 8037 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:23.808990 9133 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:23.784272 9033 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:23.834066 8044 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
W20260616 19:58:23.838629 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 136658us
I20260616 19:58:23.848480 7963 raft_consensus.cc:2410] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:23.848923 8011 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.820462 9045 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:23.834231 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.856396 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.833415 9049 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:23.858826 9343 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260616 19:58:23.867198 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.859131 8038 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:23.859823 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.868268 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:23.852128 9342 log.cc:927] Time spent T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260616 19:58:23.868908 8040 raft_consensus.cc:2368] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3.
I20260616 19:58:23.869361 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.870442 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.865541 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:23.871912 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.873699 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.874598 8085 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.868875 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.851202 8011 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.877792 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.878544 9346 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:23.878901 9230 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:23.880671 9195 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.884714 8036 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:23.889994 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.901005 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:23.894552 9323 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.806939 9098 consensus_queue.cc:237] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.914299 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:23.915068 7964 raft_consensus.cc:2368] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
I20260616 19:58:23.917158 9345 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:23.896318 8045 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:23.945408 9221 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:23.958851 9330 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.897378 9325 consensus_queue.cc:1059] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:23.850008 9350 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:23.960487 9350 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.959995 8119 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260616 19:58:23.864645 9236 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.893497 8085 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:23.896571 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 44108.2us
I20260616 19:58:23.898420 9328 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:23.905444 8042 raft_consensus.cc:1240] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:23.971028 9330 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.946038 8045 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260616 19:58:23.946591 9195 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:23.948971 7966 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:23.949569 7961 raft_consensus.cc:1275] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:23.951047 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:23.953214 7935 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:23.901939 8084 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:23.977541 7935 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:23.956909 9301 consensus_meta.cc:287] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.524s user 0.006s sys 0.005s
I20260616 19:58:23.851900 9324 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.981093 8044 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:23.981896 9324 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.928453 8041 raft_consensus.cc:2393] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:23.839663 9292 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.899082 9346 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.986186 9346 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:23.988195 9292 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:23.989363 9280 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:23.989893 9280 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:23.990340 9280 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.901973 8040 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:23.970086 9350 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:23.906949 9316 consensus_meta.cc:287] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.521s user 0.008s sys 0.001s
I20260616 19:58:23.955107 7965 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:58:23.899786 9326 consensus_queue.cc:1059] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.080s
I20260616 19:58:23.980568 9221 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:23.993777 9316 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.994517 9230 raft_consensus.cc:697] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.994345 9133 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.995977 9316 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:23.996186 9326 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:23.996608 9326 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.996711 9325 raft_consensus.cc:3055] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:23.996623 8043 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:23.997089 9325 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:23.995401 9230 consensus_queue.cc:237] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.997524 9326 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.998088 9325 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:23.999219 9325 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:23.979858 7935 leader_election.cc:400] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3.
I20260616 19:58:23.984848 8011 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:23.991045 9301 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:23.992424 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:23.994158 9221 raft_consensus.cc:697] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:23.981717 9195 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:23.957396 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 80956.5us
W20260616 19:58:24.003409 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 101542us
I20260616 19:58:23.869585 9320 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.004761 7934 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:24.000111 7935 leader_election.cc:403] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260616 19:58:24.000254 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 106772us
I20260616 19:58:24.006464 8085 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:23.998770 9326 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:24.013290 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:24.014792 7935 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:24.016649 9316 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:24.017915 7967 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:24.021389 8119 raft_consensus.cc:1275] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:24.022439 9320 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.023360 8113 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260616 19:58:24.023545 9295 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:24.025563 9323 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.025998 9182 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:24.026718 9182 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:24.027357 9195 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.002767 8946 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:24.012846 9331 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:24.029572 8946 raft_consensus.cc:697] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:24.030457 8946 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.032558 9221 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.034746 9292 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:24.035013 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:24.035604 8041 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:24.037299 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:24.039719 8043 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:23.972967 9035 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.041093 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:24.042881 8042 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:24.041069 9045 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:24.041762 8913 raft_consensus.cc:3055] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:24.045630 8913 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:24.047104 8045 raft_consensus.cc:1275] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260616 19:58:24.052950 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 87079.4us
I20260616 19:58:24.054461 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:24.054728 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:24.055908 8011 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:24.015079 9300 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:24.019440 9319 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:24.020628 9049 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:24.057870 9328 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:24.058524 8913 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.059319 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:24.060384 9350 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:24.045568 9323 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:24.061096 9351 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:24.061580 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:24.065205 8042 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
W20260616 19:58:24.065634 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 43619.8us
I20260616 19:58:24.061933 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:24.071488 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:24.075706 7961 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:24.072366 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:24.078217 8913 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:24.081816 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:24.082386 7965 raft_consensus.cc:2393] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:24.072985 9301 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:24.051630 9126 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260616 19:58:24.084916 9301 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.077370 9045 consensus_queue.cc:1059] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260616 19:58:24.027933 9182 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.086432 9045 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:24.086851 9045 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:24.082683 9323 raft_consensus.cc:697] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:24.078404 9300 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:24.112211 9281 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:24.113440 7934 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:24.071851 9295 consensus_queue.cc:1059] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260616 19:58:24.136106 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:24.112684 7935 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:24.136313 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:24.137498 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 48929.5us
I20260616 19:58:24.137885 8116 raft_consensus.cc:2368] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5.
W20260616 19:58:24.141249 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 59937.7us
I20260616 19:58:24.136781 7961 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:24.141953 8011 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:24.087658 9045 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.146885 7934 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:24.157768 9034 consensus_queue.cc:1175] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" responder_term: 4 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260616 19:58:24.092168 9349 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260616 19:58:24.167759 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:24.138864 9328 consensus_queue.cc:1059] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260616 19:58:24.137984 9301 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.138617 9351 consensus_queue.cc:1059] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260616 19:58:24.161962 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:24.140513 9282 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260616 19:58:24.159777 9300 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:24.162011 9320 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260616 19:58:24.226199 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 52357.1us
I20260616 19:58:24.180029 9350 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:24.175601 9348 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:24.186601 9323 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.363651 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 186254us
W20260616 19:58:24.527385 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 114702us
I20260616 19:58:24.534773 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:24.544579 8039 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:24.484824 9318 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:24.552253 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 412615us
I20260616 19:58:24.537552 8041 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
W20260616 19:58:24.558499 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:24.539445 9195 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:24.566754 8039 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
W20260616 19:58:24.570178 9324 consensus_meta.cc:287] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.588s user 0.016s sys 0.005s
I20260616 19:58:24.555775 9300 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:24.589318 9349 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
W20260616 19:58:24.598047 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 56943us
I20260616 19:58:24.570633 9348 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:24.634135 8913 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260616 19:58:24.573987 9292 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s user 0.007s sys 0.019s
I20260616 19:58:24.637447 8913 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260616 19:58:24.561605 9221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s user 0.002s sys 0.020s
I20260616 19:58:24.550550 9350 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.560375 9230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s user 0.003s sys 0.019s
I20260616 19:58:24.644179 8011 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:24.583216 9035 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.587121 9320 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:24.590067 9301 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:24.652839 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:24.602161 9349 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:24.642091 9292 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s user 0.007s sys 0.020s
W20260616 19:58:24.646608 9221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s user 0.002s sys 0.021s
I20260616 19:58:24.572326 9318 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.660166 9230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s user 0.004s sys 0.019s
I20260616 19:58:24.688652 8011 leader_election.cc:400] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3.
W20260616 19:58:24.773013 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 112772us
I20260616 19:58:24.774070 8011 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:24.688023 9035 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:24.697997 9328 consensus_queue.cc:1243] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260616 19:58:24.791042 7962 consensus_meta.cc:287] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.766s user 0.010s sys 0.000s
I20260616 19:58:24.791843 9385 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:24.687156 9301 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.663641 9320 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.840507 9365 log.cc:927] Time spent T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.246s user 0.000s sys 0.001s
I20260616 19:58:24.779023 9384 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:24.656108 9348 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.850255 8085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s user 0.004s sys 0.004s
I20260616 19:58:24.768282 9221 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:24.863341 8085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s user 0.005s sys 0.004s
I20260616 19:58:24.689448 9349 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.865489 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 91548.6us
W20260616 19:58:24.867842 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 728282us
W20260616 19:58:24.856052 9280 consensus_meta.cc:287] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.863s user 0.008s sys 0.008s
W20260616 19:58:24.852924 9325 consensus_meta.cc:287] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.853s user 0.014s sys 0.009s
W20260616 19:58:24.863662 9295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s user 0.016s sys 0.036s
I20260616 19:58:24.848371 9182 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:58:24.911530 9295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s user 0.017s sys 0.036s
I20260616 19:58:24.911846 9182 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.913802 9320 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.914891 9182 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:24.864730 9346 consensus_meta.cc:287] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.878s user 0.027s sys 0.000s
I20260616 19:58:24.852567 7962 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:24.873720 9384 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.930920 9384 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:24.931861 9360 log.cc:927] Time spent T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.507s user 0.001s sys 0.000s
I20260616 19:58:24.933450 8085 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:24.934090 9369 log.cc:927] Time spent T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20260616 19:58:24.936293 9295 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:24.936954 9295 raft_consensus.cc:697] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:24.938493 9295 consensus_queue.cc:237] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.941769 9361 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.806s user 0.002s sys 0.000s
I20260616 19:58:24.848563 9376 consensus_queue.cc:1059] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.126s
I20260616 19:58:24.942636 9376 consensus_queue.cc:1243] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260616 19:58:24.944061 9376 raft_consensus.cc:3055] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:24.944444 9376 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:24.945214 9376 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.946413 9376 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:24.949860 9319 consensus_meta.cc:287] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.875s user 0.007s sys 0.002s
I20260616 19:58:24.950286 9319 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3.
W20260616 19:58:24.951459 9363 log.cc:927] Time spent T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.437s user 0.000s sys 0.001s
I20260616 19:58:24.952291 9045 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:24.795815 9035 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.958182 9035 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.959931 9346 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:24.960794 9364 log.cc:927] Time spent T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.421s user 0.000s sys 0.009s
I20260616 19:58:24.962697 9346 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:24.965209 9350 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.907020 9301 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.909795 9388 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:24.969287 9388 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:24.905055 8820 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260616 19:58:24.910640 9377 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
I20260616 19:58:24.914680 9345 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:24.785984 9300 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.851217 9236 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:24.922492 9348 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.862728 9385 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:24.920370 9349 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:24.893263 9221 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:24.965981 8037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 964) took 1413 ms. Trace:
W20260616 19:58:24.926966 8011 connection.cc:256] RPC call timeout handler was delayed by -1.0574274089999562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
W20260616 19:58:25.015857 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 363076us
W20260616 19:58:25.013491 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 68641.7us
I20260616 19:58:25.019344 8037 rpcz_store.cc:276] 0616 19:58:23.552431 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:23.877386 (+324955us) service_pool.cc:224] Handling call
0616 19:58:23.877695 (+ 309us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:23.969151 (+ 91456us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:23.969192 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:23.971032 (+ 1840us) log.cc:844] Serialized 26 byte log entry
0616 19:58:24.139799 (+168767us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:24.139839 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:24.185292 (+ 45453us) spinlock_profiling.cc:243] Waited 91 ms on lock 0x7b54003acad4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:24.185338 (+ 46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:24.960042 (+774704us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:24.965721 (+ 5679us) raft_consensus.cc:1685] finished
0616 19:58:24.965750 (+ 29us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:24.965912 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":12503,"spinlock_wait_cycles":215266944,"thread_start_us":168403,"threads_started":1,"wal-append.queue_time_us":196505}
I20260616 19:58:25.027990 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.030102 7961 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:25.030189 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:25.031245 7965 raft_consensus.cc:2393] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:25.031845 7963 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:25.032526 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.037358 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.038075 7970 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:25.038991 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.039613 7970 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:25.040875 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.041638 7965 raft_consensus.cc:2393] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:25.041741 8011 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:24.980834 9320 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.042778 7965 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260616 19:58:25.049592 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.050407 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.056030 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:25.056247 7961 raft_consensus.cc:1275] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260616 19:58:25.056710 7961 pending_rounds.cc:85] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:25.056932 7961 pending_rounds.cc:107] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260616 19:58:25.057155 7961 raft_consensus.cc:2889] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:25.057561 7968 raft_consensus.cc:1240] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260616 19:58:24.972116 9388 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.060074 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:25.062916 7967 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:24.898804 9280 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.052834 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:25.069590 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:25.071979 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.018823 9326 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260616 19:58:25.076930 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.082775 8115 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:25.088819 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.096693 7966 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260616 19:58:25.076617 9328 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:25.096868 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:25.103044 7962 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 647) took 1618 ms (client timeout 1954 ms). Trace:
I20260616 19:58:25.104378 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.105038 8114 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
W20260616 19:58:25.103468 7962 rpcz_store.cc:269] 0616 19:58:23.484992 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:23.487779 (+ 2787us) service_pool.cc:224] Handling call
0616 19:58:25.102867 (+1615088us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54000f9954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:25.102971 (+ 104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897187584}
I20260616 19:58:25.107434 9236 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:25.108399 9395 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:25.108920 9397 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.109109 9294 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:25.109527 9397 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:25.114187 7963 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
W20260616 19:58:25.114925 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 73330.9us
I20260616 19:58:25.115689 8011 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:25.116554 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.117794 8011 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.120036 7969 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260616 19:58:25.076061 9325 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:25.055676 9320 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.137079 8114 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:25.112084 9397 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.144477 7934 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:25.112910 9385 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.140853 7964 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260616 19:58:25.123816 9236 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:25.130648 9383 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:25.130049 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.123672 9327 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260616 19:58:25.169163 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 110765us
I20260616 19:58:25.167410 7935 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:25.125551 9328 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:25.174384 8009 leader_election.cc:400] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5.
I20260616 19:58:25.191095 8009 leader_election.cc:403] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260616 19:58:25.191542 7935 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:25.151615 9301 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:25.194557 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.169186 9327 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:25.136092 9325 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.196545 8041 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:25.194931 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.208653 8037 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
W20260616 19:58:25.196067 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 49673.8us
I20260616 19:58:25.196183 9301 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:25.197798 9349 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.197901 9133 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.174636 9383 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:25.191673 9398 consensus_queue.cc:1059] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260616 19:58:25.194201 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.220597 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:25.211818 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 81842.9us
I20260616 19:58:25.232522 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:25.241536 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.242653 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:25.221148 9359 log.cc:927] Time spent T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.653s user 0.008s sys 0.005s
I20260616 19:58:25.250306 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.206393 9327 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:25.223608 9391 log.cc:927] Time spent T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.064s user 0.000s sys 0.001s
I20260616 19:58:25.198304 9317 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:25.221984 9398 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:25.258249 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:25.273993 8045 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:25.258471 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.274641 8118 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:25.218401 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.224678 9325 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:25.218869 9301 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.195802 9049 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:25.225143 9318 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.204953 9382 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.222149 9328 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5.
I20260616 19:58:25.277393 8085 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:25.290271 9402 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:25.290745 9402 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:25.291285 9402 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:25.291553 9394 log.cc:927] Time spent T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.092s user 0.000s sys 0.000s
I20260616 19:58:25.292230 9033 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:25.237687 9236 consensus_queue.cc:1059] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260616 19:58:25.220968 9383 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.302775 9236 raft_consensus.cc:3055] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:25.303139 9236 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:25.303952 9236 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.305126 9236 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.219353 9349 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.299113 8009 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.308034 8042 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260616 19:58:25.219926 9133 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.320740 9392 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:25.320928 9133 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.321259 9392 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:25.321691 9392 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.323487 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 988) took 1243 ms. Trace:
I20260616 19:58:25.273696 9398 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:25.293654 9381 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260616 19:58:25.323864 7966 rpcz_store.cc:276] 0616 19:58:24.079635 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:24.079755 (+ 120us) service_pool.cc:224] Handling call
0616 19:58:24.080052 (+ 297us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:25.100276 (+1020224us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:25.323254 (+222978us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400002dd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:25.323431 (+ 177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337470336}
I20260616 19:58:25.330194 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.322134 8038 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:25.327334 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.335855 7970 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:25.334439 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.337400 8042 raft_consensus.cc:2393] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:25.338313 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:25.339215 8039 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:25.298291 9317 raft_consensus.cc:697] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:25.328045 9403 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260616 19:58:25.357635 7965 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:25.360312 7966 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260616 19:58:25.360884 7966 pending_rounds.cc:85] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:25.361208 7966 pending_rounds.cc:107] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260616 19:58:25.361506 7966 raft_consensus.cc:2889] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:25.338846 9106 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.294096 9382 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.346812 9350 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:25.373474 9351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.801s user 0.022s sys 0.039s
I20260616 19:58:25.357985 9129 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.398883 9129 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.385386 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.378136 9106 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.390796 9385 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:25.391340 9400 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260616 19:58:25.392220 9347 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260616 19:58:25.393190 9387 log.cc:927] Time spent T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.112s user 0.000s sys 0.000s
I20260616 19:58:25.395152 9403 consensus_queue.cc:1243] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260616 19:58:25.404346 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:25.406090 8119 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260616 19:58:25.389477 9351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s user 0.022s sys 0.040s
I20260616 19:58:25.416664 7962 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:25.421214 7935 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:25.430297 7970 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:25.419541 9240 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.430110 9403 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.432020 9240 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.432240 9347 consensus_queue.cc:1059] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:25.433091 9347 consensus_queue.cc:1243] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260616 19:58:25.434336 9240 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.432142 9403 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.442008 9403 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:25.435426 9402 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.446147 9351 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.446672 9351 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.447183 9301 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.448386 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.448812 9351 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.449465 9349 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.450865 9382 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.452942 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:25.454782 8114 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:25.450860 9402 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:25.419070 9398 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.423925 9317 consensus_queue.cc:237] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.459534 9106 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.457415 9282 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:25.413002 9385 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.460860 8040 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260616 19:58:25.471103 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 239654us
I20260616 19:58:25.474047 7935 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:25.478596 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.502326 9381 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.486698 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.503718 9281 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.510744 9281 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.512058 9349 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.514379 9240 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.514724 9349 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.532639 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.532449 9240 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.533776 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.534392 9349 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.534605 9240 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.534770 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.535385 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.536514 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.537490 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.538034 7966 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:25.538394 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.538842 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.539105 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.540943 7969 raft_consensus.cc:2393] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:25.514444 9408 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.541287 8117 raft_consensus.cc:2410] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:25.542201 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.544229 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.545724 8009 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.508103 9398 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.549420 9377 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.551972 9182 raft_consensus.cc:3055] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
W20260616 19:58:25.564630 9378 log.cc:927] Time spent T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.759s user 0.000s sys 0.011s
I20260616 19:58:25.566210 9098 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.575198 9098 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.577415 9098 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:25.577942 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.578723 8037 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:25.579808 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.572753 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.573380 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.574445 9320 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.581090 8084 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:25.568356 7968 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260616 19:58:25.608991 8041 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:25.587671 7969 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:25.626207 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.628473 8112 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:25.615600 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.631325 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.505407 9390 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:25.632256 8041 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:25.512537 9351 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:25.634707 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 89063.4us
I20260616 19:58:25.634711 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.503883 9282 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.636471 8040 raft_consensus.cc:2393] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:25.643236 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.582067 9182 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:25.637430 9324 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:25.638890 7967 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:25.641592 9221 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.639050 9293 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:25.640797 8110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53916 (request call id 647) took 1537 ms. Trace:
I20260616 19:58:25.631263 8011 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.655702 8011 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.657189 8011 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.658893 8011 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.644042 8041 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:58:25.660836 9171 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:25.661815 7967 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.3-4.4]
I20260616 19:58:25.637930 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.644812 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:25.540088 9397 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:25.633633 9351 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.585345 9385 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.644538 9282 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:25.503459 9392 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.585639 9377 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:25.638902 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 57897.7us
I20260616 19:58:25.615983 9097 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:25.663139 7967 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260616 19:58:25.628742 9280 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.662793 7969 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:25.666288 9380 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:25.606612 9129 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.652872 9240 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.668800 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:25.543404 9327 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.732375 8038 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:25.732880 8043 raft_consensus.cc:1240] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:25.505154 9381 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.736383 9381 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.635284 9390 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.737488 9390 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:25.652647 8110 rpcz_store.cc:276] 0616 19:58:24.103437 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:24.103570 (+ 133us) service_pool.cc:224] Handling call
0616 19:58:24.103801 (+ 231us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:24.547688 (+443887us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:24.547718 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:24.554235 (+ 6517us) log.cc:844] Serialized 26 byte log entry
0616 19:58:24.663474 (+109239us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:24.663527 (+ 53us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:24.663594 (+ 67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:25.551473 (+887879us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:58:25.640415 (+ 88942us) raft_consensus.cc:1685] finished
0616 19:58:25.640446 (+ 31us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:25.640703 (+ 257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25305,"raft.queue_time_us":42921,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":197758,"spinlock_wait_cycles":122600320,"thread_start_us":515297,"threads_started":2,"wal-append.queue_time_us":237958}
I20260616 19:58:25.664721 9348 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:25.710903 7934 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:25.663823 9328 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:25.639148 9349 consensus_queue.cc:1048] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260616 19:58:25.661697 9350 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:25.661325 9171 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.716183 9385 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.725931 9392 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.690310 8036 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:25.740631 8009 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:25.666905 9383 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:25.763581 8037 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:25.764855 9240 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.711484 9097 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:25.634796 9408 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:25.775694 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:25.783970 8039 raft_consensus.cc:1275] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:25.766399 9350 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.664706 9347 consensus_queue.cc:1059] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260616 19:58:25.764266 9382 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:25.669364 9397 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:25.717151 9182 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.786636 9133 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:25.789505 9324 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:25.812940 8084 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:25.799090 9382 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:25.764489 9381 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260616 19:58:25.804585 9301 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.788786 9385 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.789306 9348 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260616 19:58:25.823849 8036 raft_consensus.cc:3055] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:25.826275 8036 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:25.815527 8820 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.815474 9182 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.831234 9098 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:25.831357 9345 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:25.814144 9097 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.815853 9324 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.817966 9382 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.835407 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.819149 9402 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:25.836166 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:25.827054 8036 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:25.849288 8036 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:25.840310 9347 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:25.853615 9097 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:25.854470 9347 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:25.855369 9408 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.892040 9171 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:25.904367 8036 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260616 19:58:25.905031 8036 pending_rounds.cc:85] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 3
I20260616 19:58:25.897324 9408 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:25.910704 9419 log.cc:927] Time spent T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.066s user 0.000s sys 0.001s
I20260616 19:58:25.910557 9408 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:25.911170 9315 raft_consensus.cc:3055] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:25.911573 9315 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:25.912361 9315 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.913059 9408 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:25.913468 9315 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:25.905288 8036 pending_rounds.cc:107] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.4
I20260616 19:58:25.922199 8036 raft_consensus.cc:2917] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Aborting config change with OpId 4.4: Aborted: Op aborted by new leader
I20260616 19:58:25.922904 8036 raft_consensus.cc:2889] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:25.925716 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:25.903860 9350 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.905719 9377 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:25.934640 9316 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260616 19:58:25.942759 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:25.947784 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:25.974654 8045 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260616 19:58:25.959286 9293 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:25.908033 9347 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.946180 9282 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:25.995368 8042 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:25.978826 9324 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:26.000988 9421 log.cc:927] Time spent T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
W20260616 19:58:26.001732 9424 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.071s user 0.000s sys 0.001s
I20260616 19:58:25.980428 9385 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:25.985625 8784 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:26.003808 9347 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:26.025437 9281 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:26.029814 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:26.034106 9315 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.038185 9315 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.038950 9390 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.040195 9315 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.041644 9282 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:26.040212 9429 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260616 19:58:26.030675 9432 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:26.044654 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:26.018463 9293 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260616 19:58:26.031332 9240 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.051571 9397 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.052495 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.053056 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.053663 8115 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:26.054502 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.055502 9395 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:26.057130 9382 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.059149 9351 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.059341 9382 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.060546 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.061156 9390 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.066220 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.067416 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.067737 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.068086 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.068869 8117 raft_consensus.cc:2410] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260616 19:58:26.069526 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:26.067783 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.081719 8110 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260616 19:58:26.083233 8039 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260616 19:58:26.083896 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.084498 8113 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.084342 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.085450 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.079521 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.087834 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.088428 8820 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260616 19:58:26.088927 8820 pending_rounds.cc:85] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Aborting all ops after (but not including) 3
I20260616 19:58:26.089295 8820 pending_rounds.cc:107] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.4
I20260616 19:58:26.089639 8820 raft_consensus.cc:2917] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Aborting config change with OpId 4.4: Aborted: Op aborted by new leader
I20260616 19:58:26.090161 8820 raft_consensus.cc:2889] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:26.088626 8039 raft_consensus.cc:2393] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:26.086922 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:26.092566 8043 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
W20260616 19:58:26.095510 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:26.095968 7962 raft_consensus.cc:1275] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:26.097443 8041 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:26.097800 9328 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.098289 9328 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.104236 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.105582 8113 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
W20260616 19:58:26.107152 9423 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.132s user 0.001s sys 0.002s
I20260616 19:58:26.125257 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:26.126530 8011 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:26.155300 8041 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260616 19:58:26.125890 8040 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
W20260616 19:58:26.157310 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 52350.5us
I20260616 19:58:26.161352 7935 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.162724 7935 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:26.162988 9281 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:26.163691 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:26.163924 9281 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:26.164484 9281 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:26.164888 9281 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.165023 7935 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.165874 9385 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:26.166230 7935 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.167035 9319 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:26.167632 9319 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:26.168313 9317 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:26.169646 8040 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260616 19:58:26.166285 9385 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:26.170104 8040 pending_rounds.cc:85] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 2
I20260616 19:58:26.170356 9385 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:26.171542 9381 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:26.172780 9381 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.173204 9381 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.174001 9281 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.175140 9381 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.175321 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:26.175843 8042 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:26.170430 8040 pending_rounds.cc:107] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260616 19:58:26.176496 8040 raft_consensus.cc:2889] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:26.177516 9281 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:26.183982 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 57575.3us
I20260616 19:58:26.184836 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:26.185494 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.155898 8041 pending_rounds.cc:85] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 1
I20260616 19:58:26.187738 8041 pending_rounds.cc:107] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260616 19:58:26.102964 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:26.188067 8041 raft_consensus.cc:2889] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:26.184175 7964 raft_consensus.cc:2393] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:26.194106 8084 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:26.198581 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:26.208519 8044 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:26.209914 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.210635 8044 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.219523 8044 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:26.222255 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.222477 7935 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:26.222826 8113 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.189841 9328 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.191099 9376 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:26.233258 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.190935 9316 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.231302 7935 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:26.187873 9349 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.239449 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.234799 8084 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:26.214154 8038 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 5.
I20260616 19:58:26.214223 9390 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.217479 9097 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:26.222132 9282 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.251890 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:26.248421 9384 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:26.247871 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.252784 9384 raft_consensus.cc:697] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:26.247272 8113 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:26.244091 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:26.196571 9336 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260616 19:58:26.255756 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.235900 9221 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:26.216054 9315 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260616 19:58:26.214351 9404 consensus_queue.cc:1059] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260616 19:58:26.260846 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:26.217808 9182 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260616 19:58:26.261732 8113 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260616 19:58:26.269902 8113 pending_rounds.cc:85] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 2
I20260616 19:58:26.270090 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.270455 8113 pending_rounds.cc:107] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260616 19:58:26.270928 8113 raft_consensus.cc:2889] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:26.264578 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:26.243947 9316 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.283051 9316 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.283447 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.285014 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.285605 8119 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.286362 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.289526 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.270772 8043 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:26.242595 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 263983us
I20260616 19:58:26.243553 9400 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260616 19:58:26.254436 9097 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:26.295367 7934 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.262909 9221 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:26.320876 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.328433 8117 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:26.322589 8112 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:26.334127 7934 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.260838 9282 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.337828 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:26.321455 9440 log.cc:927] Time spent T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.196s user 0.001s sys 0.000s
I20260616 19:58:26.325114 9294 raft_consensus.cc:3055] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:26.341005 8043 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:26.253710 9384 consensus_queue.cc:237] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:26.349428 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 60538.3us
I20260616 19:58:26.325832 9097 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.336835 9221 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:26.292131 9385 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:26.361346 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 60013.6us
I20260616 19:58:26.263839 9390 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:58:26.464648 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:26.380005 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 145243us
I20260616 19:58:26.362005 9327 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.346392 9281 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:26.495038 8084 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:26.367074 9318 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:26.368093 7934 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:26.493995 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 121472us
I20260616 19:58:26.500751 9404 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260616 19:58:26.501052 9437 log.cc:927] Time spent T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.180s user 0.000s sys 0.000s
I20260616 19:58:26.501219 9404 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:26.516431 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 148390us
I20260616 19:58:26.353403 8784 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:26.501991 9442 log.cc:927] Time spent T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.158s user 0.000s sys 0.001s
I20260616 19:58:26.361541 9294 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:26.503274 9171 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260616 19:58:26.533728 9439 log.cc:927] Time spent T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.195s user 0.006s sys 0.000s
W20260616 19:58:26.503796 9440 log.cc:927] Time spent T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.092s user 0.000s sys 0.000s
I20260616 19:58:26.508876 9182 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:26.532827 7935 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.561918 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.506287 9431 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260616 19:58:26.502594 9438 log.cc:927] Time spent T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.227s user 0.000s sys 0.000s
I20260616 19:58:26.448179 9319 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:26.538605 7934 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.431562 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53916 (request call id 654) took 1205 ms. Trace:
I20260616 19:58:26.504225 9280 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.375303 9325 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.163s
I20260616 19:58:26.535529 9404 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:26.554630 9318 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:26.537459 9381 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:26.556885 9171 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:26.569658 7934 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:26.584905 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 78967.7us
I20260616 19:58:26.565714 9436 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:26.553321 9389 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260616 19:58:26.587847 7934 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.560041 9382 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260616 19:58:26.565889 9319 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:26.574376 9317 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:26.570391 9281 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:26.572367 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 233409us
I20260616 19:58:26.610304 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:26.586939 9171 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:26.612910 8119 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:26.585350 9045 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:26.576799 9403 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:26.616593 9045 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.573328 9395 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260616 19:58:26.586779 9330 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:26.620882 8043 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260616 19:58:26.622310 9448 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.124s user 0.001s sys 0.000s
I20260616 19:58:26.624871 8008 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:26.628773 9385 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:26.629978 9441 log.cc:927] Time spent T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.264s user 0.000s sys 0.001s
I20260616 19:58:26.596946 8038 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260616 19:58:26.633140 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 71269.4us
I20260616 19:58:26.572448 9280 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.662168 7935 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.601732 9319 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:26.640290 9097 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:26.652343 9416 log.cc:927] Time spent T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260616 19:58:26.640014 9045 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:26.589646 9294 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.687810 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:26.666430 9317 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260616 19:58:26.652395 9281 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260616 19:58:26.692099 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 154152us
I20260616 19:58:26.688993 9385 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:26.696095 9385 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:26.696476 9385 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.702759 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.651772 9330 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:26.707203 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 114820us
I20260616 19:58:26.679278 9282 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.714496 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:26.719089 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.719313 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.693663 9382 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:26.694244 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:26.722703 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 134912us
I20260616 19:58:26.716820 7965 raft_consensus.cc:2376] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5. Re-sending same reply.
I20260616 19:58:26.703616 9381 raft_consensus.cc:697] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:26.728300 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.730496 7961 raft_consensus.cc:2410] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260616 19:58:26.729187 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.716941 9294 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:26.734541 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.735704 8085 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:26.736018 7966 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:26.736234 7965 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:26.665800 9318 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.724192 9097 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:26.724459 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.748103 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:26.728590 9454 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.720641 9171 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.751513 9382 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
W20260616 19:58:26.769058 9447 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.127s user 0.001s sys 0.000s
I20260616 19:58:26.774704 9461 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:26.781818 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.782199 9385 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.783016 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.757400 9454 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.784742 9382 consensus_queue.cc:1243] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260616 19:58:26.790364 9461 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:26.773986 9381 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.568243 8111 rpcz_store.cc:276] 0616 19:58:25.226587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:25.239318 (+ 12731us) service_pool.cc:224] Handling call
0616 19:58:25.294649 (+ 55331us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:25.317235 (+ 22586us) raft_consensus.cc:1534] Early marking committed up to index 3
0616 19:58:25.366884 (+ 49649us) log.cc:844] Serialized 14 byte log entry
0616 19:58:26.022274 (+655390us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b18002d6280. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c216587 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe86ee7e5ab 00007fe86ee840c3 00007fe86ee845da 00007fe86f2efe0a
0616 19:58:26.260720 (+238446us) log.cc:844] Serialized 14 byte log entry
0616 19:58:26.343557 (+ 82837us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:58:26.347046 (+ 3489us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:58:26.352580 (+ 5534us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:26.364074 (+ 11494us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:26.411631 (+ 47557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71057,"prepare.queue_time_us":32085,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":4541,"raft.queue_time_us":94530,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6331,"spinlock_wait_cycles":31970432,"thread_start_us":170200,"threads_started":1,"wal-append.queue_time_us":867245}
I20260616 19:58:26.830519 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.832947 8114 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:26.826267 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.837463 8119 raft_consensus.cc:2410] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260616 19:58:26.837855 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 76997.5us
I20260616 19:58:26.838507 8118 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:26.817432 9280 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.837206 9317 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:26.840930 9171 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:26.839519 9330 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:26.849373 9452 log.cc:927] Time spent T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.123s user 0.001s sys 0.000s
W20260616 19:58:26.852418 9460 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.062s user 0.000s sys 0.001s
I20260616 19:58:26.865577 9464 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:26.849810 9456 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260616 19:58:26.870635 9382 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:26.891471 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.867990 9317 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:26.871971 9280 consensus_queue.cc:1059] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260616 19:58:26.869632 9473 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260616 19:58:26.897823 8111 pending_rounds.cc:85] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 1
I20260616 19:58:26.911405 8118 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:26.910560 9049 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:26.916083 9465 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260616 19:58:26.885944 9464 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.917649 7934 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:26.895509 9382 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.900468 9221 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.914790 8111 pending_rounds.cc:107] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260616 19:58:26.874119 9461 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.932852 7967 raft_consensus.cc:2393] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:26.925253 9317 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:26.932325 8111 raft_consensus.cc:2889] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:26.925484 9454 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.894757 8784 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:26.917203 9280 consensus_queue.cc:1243] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260616 19:58:26.926267 9382 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.929853 9221 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.960464 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:26.923245 9472 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:26.920804 9049 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:26.954000 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 59361.1us
I20260616 19:58:26.939311 9319 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.951769 9454 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:26.968820 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.959795 9436 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:26.972404 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.968302 9472 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:26.978724 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.981061 8118 raft_consensus.cc:2410] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:58:26.975827 9464 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.982575 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:26.986912 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:26.988389 9330 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:26.988917 9330 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.989634 9319 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.989893 9390 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:26.990959 9330 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:26.981783 9477 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:26.981726 9473 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:27.003834 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:26.976213 9454 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:27.022656 9349 consensus_meta.cc:287] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.780s user 0.006s sys 0.006s
I20260616 19:58:26.935845 9404 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:26.992244 9477 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.029798 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:27.037886 8118 raft_consensus.cc:2368] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4.
I20260616 19:58:27.039217 7934 leader_election.cc:400] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4.
I20260616 19:58:27.039527 7934 leader_election.cc:403] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260616 19:58:26.978240 9472 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.026129 9349 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:27.029349 9390 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.042114 9221 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:27.007972 9473 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.064342 8041 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:27.045943 9404 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:27.072139 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 174900us
I20260616 19:58:27.079133 8113 raft_consensus.cc:2410] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:27.046473 9390 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.087805 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.090469 9473 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:27.092584 9473 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:27.087488 9049 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.102007 9404 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260616 19:58:27.153607 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:27.125873 9049 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.183274 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:27.198666 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:27.200834 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.194768 7966 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:27.167433 9454 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.208302 7965 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:27.167086 9478 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.177690 9318 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.192986 9464 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.199239 9382 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.230386 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.231742 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.238432 7969 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:27.223248 9381 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.177184 9049 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.213240 9477 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:27.225634 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.245241 7962 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:27.232424 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:27.240669 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.232954 9379 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.252209 7965 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:27.234560 9376 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.233273 9453 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:27.309173 9453 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:27.309552 9049 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.232880 9351 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.310772 9351 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.312500 9319 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.312832 9351 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.313464 9472 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.313253 9319 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.313954 9472 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:27.303201 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 678355us
I20260616 19:58:27.241346 9478 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.239491 9403 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:27.316036 9319 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:27.327446 9486 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260616 19:58:27.231345 9318 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.334091 9318 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.336236 9327 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:27.336978 9327 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.339641 9327 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.303512 7965 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260616 19:58:27.305904 9133 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260616 19:58:27.306630 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 52215.3us
I20260616 19:58:27.252003 9464 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.308401 9379 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.550501 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.315968 9472 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:27.552426 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 201464us
I20260616 19:58:27.352982 8044 pending_rounds.cc:85] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 1
I20260616 19:58:27.553040 8044 pending_rounds.cc:107] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260616 19:58:27.553254 8044 raft_consensus.cc:2889] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:27.357107 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:27.317703 9485 log.cc:927] Time spent T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.099s user 0.000s sys 0.000s
I20260616 19:58:27.349215 9403 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:27.318517 9390 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.251657 9381 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.551600 9379 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.385525 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:27.570668 8040 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:27.243242 9382 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.389122 9385 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:27.411257 9488 log.cc:927] Time spent T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
W20260616 19:58:27.412197 9487 log.cc:927] Time spent T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.116s user 0.001s sys 0.000s
I20260616 19:58:27.386653 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:27.316401 9376 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.579584 9376 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.536897 9221 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4.
I20260616 19:58:27.253801 9280 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.581327 9382 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.581677 9280 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.583659 9280 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.538775 9318 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.540745 9462 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:27.584211 9462 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:27.583812 9453 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.584872 9453 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:27.584957 9462 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.290408 9324 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.255306 9427 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.585971 9126 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.586233 9427 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.586474 9126 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.587247 8038 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:27.588253 9427 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.588938 9324 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.589185 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:27.589843 8038 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 1.
I20260616 19:58:27.590860 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:27.591070 9318 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:27.547909 7961 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260616 19:58:27.593408 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:27.563105 8043 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:27.595196 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:27.563792 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 714) took 1338 ms. Trace:
I20260616 19:58:27.566156 7933 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:27.596063 9126 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.576516 9381 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.597110 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:27.597705 8040 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:27.545260 9478 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.598453 7966 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:27.596014 8043 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:27.608695 8037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 1029) took 1429 ms. Trace:
I20260616 19:58:27.609603 8037 rpcz_store.cc:276] 0616 19:58:26.178779 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:26.183552 (+ 4773us) service_pool.cc:224] Handling call
0616 19:58:26.193783 (+ 10231us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:26.294528 (+100745us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:26.294571 (+ 43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:26.337681 (+ 43110us) log.cc:844] Serialized 26 byte log entry
0616 19:58:26.579561 (+241880us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:26.590402 (+ 10841us) log.cc:844] Serialized 14 byte log entry
0616 19:58:26.590618 (+ 216us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:27.223018 (+632400us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b180034c3c0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c1a7e1a 00007fe8758b35d9 00007fe8731bca6f 00007fe86c233bb1 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d
0616 19:58:27.564916 (+341898us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b180034c3c0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:27.608369 (+ 43453us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b54003aa554. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:27.608421 (+ 52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:27.608473 (+ 52us) raft_consensus.cc:1685] finished
0616 19:58:27.608490 (+ 17us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:27.608636 (+ 146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1151,"prepare.queue_time_us":8797,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":6376,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":29894,"spinlock_wait_cycles":268113536,"thread_start_us":131812,"threads_started":1,"wal-append.queue_time_us":260268}
W20260616 19:58:27.599368 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 46261.9us
I20260616 19:58:27.591459 8038 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:27.618026 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 83647.3us
I20260616 19:58:27.597344 8045 rpcz_store.cc:276] 0616 19:58:26.225168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:26.624790 (+399622us) service_pool.cc:224] Handling call
0616 19:58:26.625119 (+ 329us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:27.225496 (+600377us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:27.225545 (+ 49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:27.230459 (+ 4914us) log.cc:844] Serialized 26 byte log entry
0616 19:58:27.522137 (+291678us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:58:27.522245 (+ 108us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:27.522346 (+ 101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:27.563583 (+ 41237us) raft_consensus.cc:1685] finished
0616 19:58:27.563598 (+ 15us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:27.563736 (+ 138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":668786,"raft.queue_time_us":600049,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4942,"thread_start_us":211530,"threads_started":1,"wal-append.queue_time_us":306596}
W20260616 19:58:27.629741 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:27.632920 9385 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:27.639590 8011 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:27.644100 8011 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:27.648397 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:27.590863 9324 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.609021 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.630542 9379 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:27.624637 9376 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.666442 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:27.631160 9427 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.641986 9383 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:27.642577 9377 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.642743 9351 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.671996 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.673956 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:27.681202 8116 raft_consensus.cc:2410] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:27.676126 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.678115 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.642369 9327 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:27.654721 9455 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:27.685513 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.667356 8040 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:27.691447 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.695966 8114 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:27.690421 8118 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:27.686780 8011 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:27.667778 9324 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:27.668319 9133 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:27.677085 9328 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:58:27.658596 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 92615.8us
W20260616 19:58:27.713143 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44392.6us
I20260616 19:58:27.724267 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.685021 9469 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:27.715956 9282 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.735777 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.732028 9282 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.736941 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.736879 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.738088 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.738668 9282 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:27.738929 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:27.741422 9318 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:27.736426 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.695387 9098 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:27.696925 9171 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:27.671885 9379 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.730046 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:27.730846 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 111397us
W20260616 19:58:27.715036 9504 log.cc:927] Time spent T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.067s user 0.001s sys 0.000s
I20260616 19:58:27.704905 9319 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:58:27.755426 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:27.679559 9376 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.744588 9469 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.751617 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:27.753022 8114 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:27.754024 8111 raft_consensus.cc:1217] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-5.3] Dedup: 3.2->[5.3-5.3]
I20260616 19:58:27.684866 9427 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.756107 8115 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:27.688868 9327 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.760131 7968 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:27.760445 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.761045 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.761078 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.771165 7963 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:58:27.736008 9464 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.688393 9383 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.688578 9351 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.773758 9351 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.688958 9377 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.775588 9377 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.730610 9315 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:27.777103 9315 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.777499 9315 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:27.767983 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.780421 8117 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:27.782305 9383 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.798035 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.798750 7966 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:27.800693 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.801093 9383 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:27.801357 7961 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:27.803535 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.801736 9383 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.757277 9451 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.824463 8009 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:27.767724 9381 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.767926 9221 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.783942 9468 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.768190 9330 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:27.755486 9098 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.758487 9455 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.069s
I20260616 19:58:27.762953 9319 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.756208 9171 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.830452 8009 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:27.814591 9377 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.826771 8111 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260616 19:58:27.831533 9324 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:27.846426 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.861568 9376 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.834906 9133 consensus_queue.cc:1059] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260616 19:58:27.834388 9451 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.872920 9133 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:27.873826 9453 consensus_queue.cc:1059] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:27.834802 9381 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.881578 9379 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.882190 9319 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.883067 9381 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.837582 9221 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.885418 9221 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.837925 9330 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.847918 9349 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:27.886029 9349 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:27.886427 9349 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:27.887215 9330 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:27.878923 9182 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260616 19:58:27.870755 9384 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:27.890926 9384 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.904384 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.910226 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:27.911734 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:27.914113 8043 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:27.845510 9377 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.917673 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:27.880424 9133 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:27.920394 8043 raft_consensus.cc:2410] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:27.837462 9468 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.883075 9098 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:27.921176 9182 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
W20260616 19:58:27.926070 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 85342.8us
I20260616 19:58:27.908910 9346 raft_consensus.cc:3055] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:27.912287 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 693) took 1164 ms. Trace:
I20260616 19:58:27.928627 8085 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:27.884090 9451 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.929896 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:27.940006 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.942505 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.913722 9402 consensus_queue.cc:1059] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260616 19:58:27.934309 9505 log.cc:927] Time spent T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.064s user 0.002s sys 0.000s
I20260616 19:58:27.939509 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.946413 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.938875 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.951010 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:27.927578 9182 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:27.957116 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:27.929039 9171 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:27.942345 9325 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:27.930172 9346 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:27.962462 8111 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:27.938493 9379 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260616 19:58:27.975260 8041 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 704) took 1783 ms (client timeout 1631 ms). Trace:
I20260616 19:58:27.978259 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:27.957680 9427 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:27.975703 8041 rpcz_store.cc:269] 0616 19:58:26.185803 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:26.233015 (+ 47212us) service_pool.cc:224] Handling call
0616 19:58:27.968837 (+1735822us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540023d2d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:58:27.969103 (+ 266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1869933824}
I20260616 19:58:27.961383 9469 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:27.974148 9383 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.969389 9327 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.972759 9379 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:27.991756 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:27.993089 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 306385us
I20260616 19:58:27.993101 9379 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:27.994622 9182 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.996052 9182 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:27.979074 9325 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:27.999796 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:28.002065 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.002802 7962 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:28.005003 9462 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:28.009785 9379 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.012714 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.014217 8116 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:28.015192 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.015403 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.015936 8116 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:28.016598 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.016834 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.017746 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.016636 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.018545 8113 raft_consensus.cc:2410] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:58:28.021044 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.022729 9379 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:27.999580 8011 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:28.025911 8011 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.027029 9379 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:28.029516 9327 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:27.993425 9494 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.683s user 0.005s sys 0.002s
I20260616 19:58:28.032325 8011 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.028467 9379 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:27.986620 8038 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:28.035768 9327 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:28.036970 8011 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:28.029950 9510 log.cc:927] Time spent T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.100s user 0.000s sys 0.000s
I20260616 19:58:28.030400 9346 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:27.964457 9133 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.058530 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.061024 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.060523 8011 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.002072 9295 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:28.036423 9379 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:28.041352 9515 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:28.086265 9384 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.087308 9379 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.088666 9098 raft_consensus.cc:3055] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:28.089100 9098 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:28.090523 9379 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.071991 9383 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:28.042450 9468 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:28.073607 9382 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:28.074244 9463 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:28.075089 9377 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.076085 9133 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:28.080554 9506 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.457s user 0.003s sys 0.001s
I20260616 19:58:28.081101 9390 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:28.106593 9383 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:28.090018 9098 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.108089 9098 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:28.111209 7966 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:28.042187 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.071164 8041 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 6: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:28.094868 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.120867 8111 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:28.124572 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.128017 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:28.128341 9377 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:28.128823 9377 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.129139 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.129935 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.130153 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.131672 9377 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.136801 8040 raft_consensus.cc:2410] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:58:28.144500 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.145323 9379 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:28.143591 9133 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:28.062194 9315 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.146893 8011 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:28.147692 9379 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:27.998304 9403 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.148027 8011 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:28.146215 9133 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.151751 8011 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:28.153020 7935 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.631s (SENT) [suppressed 10 similar messages]
I20260616 19:58:28.097388 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:28.148923 9315 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.186131 7963 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:28.010848 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:28.131618 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:58:28.188086 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 90769.2us
I20260616 19:58:28.191068 8117 raft_consensus.cc:1240] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:28.148082 9379 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:28.153503 9346 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:28.201043 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55744 (request call id 128) took 6074 ms. Trace:
I20260616 19:58:28.155687 9324 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:28.201457 8009 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:28.098100 9295 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:28.203004 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:28.202968 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.196924 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.211198 8041 raft_consensus.cc:2410] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260616 19:58:28.205747 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:28.041858 9325 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.207814 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.213940 7969 raft_consensus.cc:1240] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:28.203953 7866 rpcz_store.cc:276] 0616 19:58:22.122544 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:22.122772 (+ 228us) service_pool.cc:224] Handling call
0616 19:58:22.299970 (+177198us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:28.196515 (+5896545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:23.805381 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:23.894939 (+ 89558us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:23.894972 (+ 33us) write_op.cc:435] Acquired schema lock
0616 19:58:23.894989 (+ 17us) tablet.cc:662] Decoding operations
0616 19:58:23.958778 (+ 63789us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:23.958856 (+ 78us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:23.958882 (+ 26us) tablet.cc:685] Acquiring locks for 10 operations
0616 19:58:23.959728 (+ 846us) tablet.cc:701] Row locks acquired
0616 19:58:23.959748 (+ 20us) write_op.cc:260] PREPARE: finished
0616 19:58:23.960300 (+ 552us) write_op.cc:270] Start()
0616 19:58:23.960420 (+ 120us) write_op.cc:276] Timestamp: P: 1781639903960249 usec, L: 0
0616 19:58:23.960442 (+ 22us) op_driver.cc:348] REPLICATION: starting
0616 19:58:23.961077 (+ 635us) log.cc:844] Serialized 4033 byte log entry
0616 19:58:24.075889 (+114812us) op_driver.cc:464] REPLICATION: finished
0616 19:58:24.648392 (+572503us) write_op.cc:301] APPLY: starting
0616 19:58:24.648484 (+ 92us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:24.648791 (+ 307us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:24.648971 (+ 180us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:24.650334 (+ 1363us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:24.650549 (+ 215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0616 19:58:24.650595 (+ 46us) write_op.cc:312] APPLY: finished
0616 19:58:24.650902 (+ 307us) log.cc:844] Serialized 76 byte log entry
0616 19:58:24.651088 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:24.651212 (+ 124us) write_op.cc:454] Released schema lock
0616 19:58:24.651257 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:24.659513 (+ 8256us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:24.660701 (+ 1188us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":42928,"reported_tablets":22,"spinlock_wait_cycles":362306816,"child_traces":[["op",{"apply.queue_time_us":570537,"apply.run_cpu_time_us":3305,"apply.run_wall_time_us":11074,"num_ops":10,"prepare.queue_time_us":307653,"prepare.run_cpu_time_us":5627,"prepare.run_wall_time_us":179669,"replication_time_us":85263,"thread_start_us":314529,"threads_started":2}]]}
I20260616 19:58:27.931115 7964 rpcz_store.cc:276] 0616 19:58:26.744872 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:26.749452 (+ 4580us) service_pool.cc:224] Handling call
0616 19:58:26.768060 (+ 18608us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:58:26.870089 (+102029us) raft_consensus.cc:1534] Early marking committed up to index 2
0616 19:58:27.127877 (+257788us) log.cc:844] Serialized 14 byte log entry
0616 19:58:27.675470 (+547593us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:58:27.731659 (+ 56189us) log.cc:844] Serialized 26 byte log entry
0616 19:58:27.764452 (+ 32793us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:58:27.841373 (+ 76921us) log.cc:844] Serialized 14 byte log entry
0616 19:58:27.867352 (+ 25979us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:27.870679 (+ 3327us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:27.878245 (+ 7566us) raft_consensus.cc:1685] finished
0616 19:58:27.878265 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:27.908847 (+ 30582us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":197872,"prepare.queue_time_us":399382,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":3504,"raft.queue_time_us":217641,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":40063,"spinlock_wait_cycles":1201408,"thread_start_us":537290,"threads_started":3,"wal-append.queue_time_us":411765}
I20260616 19:58:28.162632 9377 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:28.209993 7934 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:28.219766 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.220327 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.194317 9390 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:28.224989 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.224857 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:28.200388 9379 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:28.216259 8038 raft_consensus.cc:2368] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4.
W20260616 19:58:28.225912 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 72919.5us
I20260616 19:58:28.229856 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.232118 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.232872 7970 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:28.232899 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:28.215750 9383 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:28.207415 9324 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:28.238435 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.240041 8117 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:28.247676 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.249122 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.251241 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.203105 9349 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.254630 7970 raft_consensus.cc:2393] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 2: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:28.255506 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.229339 9454 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:28.280893 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.309389 9454 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:28.240057 9383 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:28.245153 9324 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:28.289378 9520 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20260616 19:58:28.312969 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 55054us
W20260616 19:58:28.338110 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 107484us
I20260616 19:58:28.326931 8043 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:28.372648 9349 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.326149 9480 consensus_queue.cc:1059] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:28.400097 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.381347 8085 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:28.406924 9451 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:28.407732 9451 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.410794 9451 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.374488 9383 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:28.375741 9454 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:28.453265 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 72590.6us
I20260616 19:58:28.459335 9480 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260616 19:58:28.460593 9295 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:28.458091 9395 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:28.468348 9325 consensus_queue.cc:1059] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260616 19:58:28.478061 9455 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:28.479156 9133 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.457711 9349 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:28.485706 9455 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.461510 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.487259 9349 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.495937 8043 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:28.484935 9395 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:28.475648 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:28.496706 9182 consensus_meta.cc:287] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.500s user 0.011s sys 0.000s
I20260616 19:58:28.507030 8044 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:28.508381 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.505048 9395 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:28.535799 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:28.542084 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.542999 8042 raft_consensus.cc:2393] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:28.544651 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:28.537130 9462 consensus_meta.cc:287] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.530s user 0.011s sys 0.000s
I20260616 19:58:28.558163 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.561472 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.565884 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.557595 9349 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.593101 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.557886 9477 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260616 19:58:28.598409 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:28.582796 8038 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:28.564836 9404 consensus_queue.cc:1059] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260616 19:58:28.604552 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.631289 8119 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
W20260616 19:58:28.605990 9523 log.cc:927] Time spent T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.076s user 0.001s sys 0.000s
I20260616 19:58:28.612897 9383 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.636579 9455 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.638099 9324 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.619753 9463 raft_consensus.cc:3055] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:28.642928 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.644471 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.644589 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.621685 9349 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:28.640323 8044 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260616 19:58:28.646778 8044 pending_rounds.cc:85] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 2
I20260616 19:58:28.647086 8044 pending_rounds.cc:107] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260616 19:58:28.647369 8044 raft_consensus.cc:2889] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:28.641220 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.642107 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:28.643617 9518 log.cc:927] Time spent T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.270s user 0.006s sys 0.000s
I20260616 19:58:28.638810 9379 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.649039 8043 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:28.656364 8037 raft_consensus.cc:2368] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:28.659276 8041 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:28.646080 9349 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:28.663831 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:28.665757 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 96396.2us
I20260616 19:58:28.644038 9463 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:28.687348 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:28.694073 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.696942 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.697008 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 25) took 5276 ms. Trace:
I20260616 19:58:28.711541 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:28.712121 8118 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 6
I20260616 19:58:28.703992 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.701565 9324 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.728804 8037 raft_consensus.cc:1240] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:28.729169 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.729839 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.729949 8042 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:28.731168 8044 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:28.731071 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.731812 8042 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:28.732079 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.732842 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.733379 8042 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:28.733589 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.725345 9294 raft_consensus.cc:1064] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:28.741909 8118 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:28.722257 9383 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:28.745769 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 82088.2us
I20260616 19:58:28.710253 9463 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.749660 8084 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.707984 7864 rpcz_store.cc:276] 0616 19:58:23.420678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:23.420877 (+ 199us) service_pool.cc:224] Handling call
0616 19:58:23.549312 (+128435us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b5400002490. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6c02 00007fe87513a659 00007fe8750d28b6 00007fe8750a3968 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc
0616 19:58:28.696891 (+5147579us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3215912,"reported_tablets":1,"spinlock_wait_cycles":18735872}
I20260616 19:58:28.756596 8084 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.755466 9403 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:28.762611 9468 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:28.758888 9294 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:28.769186 9468 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:28.769758 9468 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:28.764680 9349 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.765113 9463 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:28.789184 8084 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:28.792871 8084 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.802248 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.803637 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.804589 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:28.805297 7965 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:28.806494 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:28.807931 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.809073 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.810113 7964 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:28.795547 9403 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:28.797237 9293 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:28.828457 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:28.829562 7970 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 5
I20260616 19:58:28.831434 8011 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.833562 8011 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.835292 8011 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.847584 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.847999 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:28.848852 8043 raft_consensus.cc:2376] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5. Re-sending same reply.
I20260616 19:58:28.858424 8084 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.860589 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.861846 7963 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:28.862679 8011 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.864940 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:28.878053 7965 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:28.878876 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:28.879756 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.880920 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:28.882398 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:28.882999 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:28.883914 7970 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:28.825767 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:28.887390 7935 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:28.836922 9515 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:28.844096 9324 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:28.837625 9293 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:28.850067 9294 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:28.849860 9133 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:28.864650 9390 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:28.886453 9383 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:28.889205 9351 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:28.889853 9462 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:28.894335 9293 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:28.912703 7970 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:58:28.899251 9294 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:28.919246 9294 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:28.920663 9539 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.108s user 0.000s sys 0.000s
I20260616 19:58:28.909308 9480 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260616 19:58:28.898002 9531 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:28.915645 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 721) took 1027 ms. Trace:
I20260616 19:58:28.929222 8045 rpcz_store.cc:276] 0616 19:58:27.888665 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:27.932616 (+ 43951us) service_pool.cc:224] Handling call
0616 19:58:27.932843 (+ 227us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:58:27.933007 (+ 164us) raft_consensus.cc:1534] Early marking committed up to index 3
0616 19:58:27.999617 (+ 66610us) log.cc:844] Serialized 14 byte log entry
0616 19:58:28.493459 (+493842us) spinlock_profiling.cc:243] Waited 94.1 ms on lock 0x7b1800047f20. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c216587 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe86ee7e5ab 00007fe86ee840c3 00007fe86ee845da 00007fe86f2efe0a
0616 19:58:28.801748 (+308289us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b1800047f20. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c216587 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c230a95 00007fe872cdf247 00007fe86f2efd87 00007fe86f2ff49f 00007fe86f2ff418
0616 19:58:28.910306 (+108558us) log.cc:844] Serialized 14 byte log entry
0616 19:58:28.915361 (+ 5055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:58:28.915407 (+ 46us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:58:28.915447 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:28.915507 (+ 60us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:28.915600 (+ 93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4466,"prepare.queue_time_us":281346,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":1050,"raft.queue_time_us":5157,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":5962,"spinlock_wait_cycles":219580160,"thread_start_us":172852,"threads_started":2,"wal-append.queue_time_us":685744}
I20260616 19:58:28.933506 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:28.913251 9390 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.934291 8116 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:28.910531 9476 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:28.894951 9454 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:28.944201 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 57535.2us
I20260616 19:58:28.909744 9395 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:28.940965 7868 catalog_manager.cc:5696] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:28.971818 7868 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:28.975584 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:28.975162 7868 catalog_manager.cc:5696] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 2 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:28.976687 7868 catalog_manager.cc:5696] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 1 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:28.978005 9395 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:28.940105 9390 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.974133 7934 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:28.973640 9171 raft_consensus.cc:3055] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:28.998971 9384 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:28.999536 9461 raft_consensus.cc:3055] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:28.944779 9531 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:28.948892 8817 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:58:29.001551 9454 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:28.968575 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:29.010416 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 113511us
I20260616 19:58:29.011157 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.033476 9451 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.002481 9294 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.037092 9477 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:29.037937 9477 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.039021 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.039769 7963 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:29.041461 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.042060 7963 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:29.042814 7969 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:29.043207 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.037225 9294 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.052783 7935 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.002795 9171 raft_consensus.cc:740] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:29.053937 7935 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.007750 9384 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:29.059000 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:29.055608 9477 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.065856 9550 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:29.072567 9293 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.074625 9293 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.075567 9171 consensus_queue.cc:260] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.077021 9171 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:29.081754 9531 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.057726 9462 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.059855 9384 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:29.089849 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:29.059729 9455 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.093189 9379 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.099772 8009 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:29.100973 9515 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:29.101368 9515 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.101714 9515 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:29.054945 9377 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.102675 9377 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.104750 9377 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.009923 9461 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:29.021389 9527 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260616 19:58:29.026538 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.061470 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.065091 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.108112 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:29.115018 9461 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.124509 9379 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:29.124830 9461 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:29.125054 9379 raft_consensus.cc:697] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:29.125967 9379 consensus_queue.cc:237] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.117254 9351 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:29.128409 9383 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.128706 9351 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.128909 9383 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.130734 9351 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.117017 9468 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.131232 9349 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.131737 9349 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.132740 9468 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.114317 7962 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:29.133699 9349 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.134306 8119 raft_consensus.cc:1240] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
W20260616 19:58:29.117873 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 52834.2us
I20260616 19:58:29.136034 7935 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.137197 7935 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.137933 8115 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:29.139609 9476 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.140733 7935 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.141400 9476 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.118063 9455 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:29.137933 9324 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.130762 9383 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.142743 9324 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.145035 9324 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.147730 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.138585 9377 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.149348 9377 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.150772 9349 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.151499 9377 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.151345 9349 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.153445 9349 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.154078 9383 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.154568 9383 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.156852 9383 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.157716 9133 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.158911 9383 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.159420 9383 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.160319 9324 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.160915 9324 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.164505 9324 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.165045 9351 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.165621 9351 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.168295 9221 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.170465 9383 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:29.172822 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 50750.6us
I20260616 19:58:29.173178 7935 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.174382 7935 leader_election.cc:400] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4.
I20260616 19:58:29.175141 7935 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.176306 7935 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.167969 9351 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.177016 9455 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.177469 9221 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.199563 9476 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:29.206954 9349 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.220592 9351 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.256987 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.236675 9383 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.237424 7962 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:29.267107 7935 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.274801 7935 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.265249 9476 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:29.250527 9221 consensus_queue.cc:1059] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260616 19:58:29.275929 7935 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.266350 9351 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.262274 9455 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260616 19:58:29.277827 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.261986 9349 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.275084 9462 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:29.268493 9384 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.268893 9383 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:29.275933 9390 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.285032 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.277827 9451 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.287957 7935 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.288209 9480 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:29.291607 7935 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:29.293408 9351 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.296124 9383 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:29.297648 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:29.302801 7935 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.285979 9476 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.310709 7935 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.310838 9383 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:29.311406 9383 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.313743 9390 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.302948 9221 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260616 19:58:29.318264 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 253555us
I20260616 19:58:29.317201 9454 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.314647 9384 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.321954 9550 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260616 19:58:29.324216 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 150695us
I20260616 19:58:29.324723 9385 raft_consensus.cc:3055] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:29.325101 9385 raft_consensus.cc:740] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:29.325681 8085 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:29.325927 9385 consensus_queue.cc:260] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.326519 8085 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:29.326668 9171 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:29.327035 9171 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:29.327129 9385 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:29.327268 9294 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:29.327448 9171 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:29.327874 9294 raft_consensus.cc:697] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:29.314481 9455 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260616 19:58:29.321964 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:29.328568 9455 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.329171 9455 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:29.328733 9294 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.324299 9461 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260616 19:58:29.324368 7935 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.333871 9550 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:29.334867 8085 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:29.289803 9515 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.335925 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:29.308804 9462 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260616 19:58:29.338100 7935 leader_election.cc:400] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:29.338465 7935 leader_election.cc:403] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:29.338518 9462 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:29.338973 9462 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.339339 9390 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:29.339352 9462 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:29.336896 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.336860 9515 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.342578 9390 consensus_queue.cc:1059] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:29.343838 9390 raft_consensus.cc:3055] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:29.344245 9390 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:29.344471 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.345243 8111 raft_consensus.cc:2368] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:29.345167 9390 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.345464 9293 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:29.346159 9325 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:29.346385 9390 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:29.346662 9325 raft_consensus.cc:697] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:29.348131 9462 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.348450 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.347536 9325 consensus_queue.cc:237] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.349016 8119 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:29.349382 9349 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.351783 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.352684 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.354045 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.354202 9529 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:29.354609 8114 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:29.354668 9529 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:29.355209 9529 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:29.321264 9468 consensus_queue.cc:1059] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:29.354045 8084 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.356699 9468 raft_consensus.cc:3055] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:29.356925 9477 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:29.357069 9468 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:29.358191 8084 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.357972 9468 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.358879 9477 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:29.359174 9468 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:29.359228 9477 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:29.359462 8119 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:29.359833 9477 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:29.360788 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.360949 9221 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:29.363008 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.364451 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.364987 8117 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:29.365386 7935 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.365703 9383 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.366394 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.366659 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.367193 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.367466 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.368335 9221 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:29.371184 8114 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:29.374765 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.375806 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.376299 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.376680 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.377451 8112 raft_consensus.cc:2368] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:29.378161 9346 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:29.380906 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.383667 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.379349 8084 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.385556 9462 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.386229 7965 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:29.388327 9346 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:29.394526 7967 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:29.394907 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.396395 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.396180 7934 leader_election.cc:400] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:29.394479 9171 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.397444 7934 leader_election.cc:403] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260616 19:58:29.398285 9221 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:29.404500 9098 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:29.405089 9349 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.402190 9346 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:29.405673 9349 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:29.387897 8084 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.378407 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.407473 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.408418 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.408303 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.410288 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:29.411623 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.413900 8043 raft_consensus.cc:2393] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:29.414733 8011 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.416038 8011 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.418078 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.418812 8036 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 4
I20260616 19:58:29.417431 8011 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.405768 9346 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.424669 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.427690 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.442999 8043 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:29.424624 7934 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:29.425876 8009 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:29.453747 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 65941.2us
I20260616 19:58:29.446786 8112 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:29.485076 8110 raft_consensus.cc:2368] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6.
I20260616 19:58:29.454180 9324 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:29.424134 9515 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:29.421885 9351 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.505424 9171 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.505613 9346 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.427661 9563 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:29.482442 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:29.414029 9455 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.508425 9477 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.482038 9480 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:29.481882 9531 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.467016 8036 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:29.486733 9221 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:29.507208 9529 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.540717 8043 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:29.452951 9476 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:29.458611 9560 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:29.544411 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.499747 9133 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.552510 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.568271 7965 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:29.511883 9379 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:29.486050 9324 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.556509 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.558899 7964 raft_consensus.cc:1240] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:29.543424 9550 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:29.562279 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:29.552075 9520 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260616 19:58:29.567253 7967 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:58:29.568295 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 140711us
I20260616 19:58:29.584852 8011 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.586846 8011 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.580638 8036 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:29.591408 8043 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:29.596686 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.597738 8043 raft_consensus.cc:2376] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5. Re-sending same reply.
I20260616 19:58:29.599589 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:29.596750 8085 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:29.601583 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.624200 8011 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.559877 9480 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:29.633431 8011 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.634641 8011 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.635077 9529 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.635540 9573 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:29.636271 9573 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.636823 9573 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.628331 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.613497 8036 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:29.647647 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.648238 8036 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:29.655563 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.656792 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.657760 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.658696 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.565650 9382 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.666476 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.667725 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.671546 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.673416 8036 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:29.560791 9221 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.675660 8043 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:29.623245 9351 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:29.677769 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.679608 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.681207 7970 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:29.681740 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.682135 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.683169 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.660897 9325 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:29.688378 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.650393 9560 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:29.665416 9379 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:29.685400 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.685907 8043 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:29.686439 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:29.694397 8084 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.669189 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:29.688931 7965 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:29.662209 7962 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:29.696065 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:29.691532 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 265714us
I20260616 19:58:29.687013 8036 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260616 19:58:29.694630 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 234889us
I20260616 19:58:29.695406 7935 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.695775 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:29.671989 9455 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.698781 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.706746 8110 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:29.707940 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:29.623471 9550 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.687886 9325 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:29.568744 9476 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.711550 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.714721 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.715356 8036 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 6
I20260616 19:58:29.698745 8009 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.672565 9382 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.716789 8009 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:29.721799 9456 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:29.722020 8084 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.722478 8009 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:29.723503 9572 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:29.722790 9549 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:29.724399 9572 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:29.724623 9549 raft_consensus.cc:697] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:29.724793 9572 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:29.725525 9549 consensus_queue.cc:237] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.716394 9098 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:29.731597 9098 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:29.732045 9567 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.071s user 0.000s sys 0.000s
I20260616 19:58:29.732482 9098 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.733469 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:29.734107 8045 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:29.735953 8045 raft_consensus.cc:1240] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:29.736050 8036 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:29.738575 7961 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:29.742993 8009 leader_election.cc:400] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:29.743309 8009 leader_election.cc:403] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:29.708204 9351 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.715471 9325 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:29.731590 9564 log.cc:927] Time spent T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.178s user 0.000s sys 0.001s
I20260616 19:58:29.753327 7963 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:29.718467 9515 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:29.735347 9349 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.781914 9550 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.782521 9550 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:29.782979 9550 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:29.783306 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:29.784049 8110 raft_consensus.cc:606] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:29.784850 9221 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.785410 9221 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:29.761346 9573 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:29.787839 9324 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.783831 9550 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.772207 9477 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:29.817461 9568 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4.
I20260616 19:58:29.757468 9351 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:29.814643 9562 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.252s user 0.000s sys 0.001s
I20260616 19:58:29.741202 9480 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.827219 9569 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:29.841481 9569 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.849969 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:29.855767 8042 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
W20260616 19:58:29.867936 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 125053us
I20260616 19:58:29.875260 8042 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: [4.3-5.4]
I20260616 19:58:29.760803 9379 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:29.850214 9390 consensus_meta.cc:287] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.503s user 0.011s sys 0.000s
I20260616 19:58:29.869349 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:29.892866 8009 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:29.895010 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:29.848811 9382 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.846729 9476 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.899225 8009 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:29.844115 9569 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.906865 7969 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:29.921911 7967 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:29.941381 9569 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:29.941613 8009 leader_election.cc:400] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6.
I20260616 19:58:29.941917 8009 leader_election.cc:403] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:29.941963 9569 raft_consensus.cc:697] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:29.942893 9569 consensus_queue.cc:237] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.943816 9572 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:29.946341 9572 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.947134 9572 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6.
I20260616 19:58:29.907692 9324 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:29.968034 9577 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.254s user 0.001s sys 0.000s
I20260616 19:58:29.984915 9351 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:29.984774 9294 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 2 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:29.899885 9515 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:30.049909 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.050868 8112 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
W20260616 19:58:29.995230 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 56195.6us
I20260616 19:58:29.996201 9382 consensus_queue.cc:1059] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260616 19:58:29.940646 9579 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
I20260616 19:58:30.006498 9563 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260616 19:58:30.007382 9565 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.331s user 0.000s sys 0.002s
I20260616 19:58:30.009373 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:30.010310 9325 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.030584 8044 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:30.099401 7969 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:30.035883 8110 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:30.100142 7964 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260616 19:58:30.044035 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 348673us
I20260616 19:58:30.049947 8043 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:30.056452 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.056607 8114 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:30.057693 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.104110 8112 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:30.058187 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.059158 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.062597 7967 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:30.063365 8085 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:30.064990 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.065600 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.067183 8045 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:30.072759 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:30.093479 8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:30.096719 7934 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:30.104022 7965 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:30.108781 8117 raft_consensus.cc:606] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:30.110946 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:30.121457 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 48783.4us
I20260616 19:58:30.126314 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:30.114881 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:30.137051 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:30.121793 9384 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:30.145555 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:30.104964 9585 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:30.096966 9549 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260616 19:58:30.120858 9531 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:30.120759 9589 raft_consensus.cc:3055] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:30.126943 9293 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:58:30.147444 8085 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:30.126852 9588 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:30.149246 8036 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:30.147176 8038 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:30.150204 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:30.148675 8111 raft_consensus.cc:2393] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:30.104892 9481 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.069s
I20260616 19:58:30.158737 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.159173 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.120481 9477 consensus_queue.cc:1059] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260616 19:58:30.162599 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.176640 7967 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:30.148916 9384 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:30.173540 7965 raft_consensus.cc:2393] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:30.156569 9589 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:30.177649 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.122691 9574 consensus_queue.cc:1059] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.068s
I20260616 19:58:30.178853 9384 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:30.175719 9468 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:30.184684 9325 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:30.161923 9531 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.188269 9531 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.188655 9384 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.191138 9384 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.157121 9588 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.191637 9589 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.194526 9589 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:30.191962 9295 consensus_queue.cc:1059] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260616 19:58:30.149492 9585 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.198405 9599 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:30.198930 9599 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:30.197876 9456 raft_consensus.cc:3055] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:30.200301 9456 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.201023 9599 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.201171 9456 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.202932 9456 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:30.203928 9574 raft_consensus.cc:3055] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:30.204782 9468 raft_consensus.cc:697] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.205163 9574 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.205766 9468 consensus_queue.cc:237] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.205938 9574 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.209658 9570 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:30.209031 8817 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:30.210184 9570 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.207069 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:30.212924 9549 raft_consensus.cc:3055] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:30.199884 9585 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.196040 9588 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.214792 9574 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:30.215253 9549 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:58:30.214675 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 63076.5us
I20260616 19:58:30.221524 9549 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.229508 9602 consensus_queue.cc:1059] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:30.230047 9549 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:30.230572 9455 consensus_queue.cc:1059] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:30.231415 9455 raft_consensus.cc:3055] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:30.231748 9455 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:30.232560 9455 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.233817 9455 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:30.235302 9346 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:30.229018 9098 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260616 19:58:30.236918 9098 consensus_queue.cc:1243] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260616 19:58:30.237404 8112 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:30.236467 9570 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.240576 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:30.216267 7962 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:30.238317 7934 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:30.241142 8112 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:30.255673 8112 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:30.256203 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.256875 8117 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:30.263090 7934 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:30.265329 7934 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:30.261761 9454 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:30.270341 9462 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:30.238798 9456 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260616 19:58:30.268051 9480 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:30.272085 9454 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:30.275166 9462 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:30.280745 9454 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:30.288825 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.288829 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.282370 9480 raft_consensus.cc:697] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:30.285787 9462 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:30.279678 9182 consensus_queue.cc:1059] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260616 19:58:30.297823 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:30.297744 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.303331 9599 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:30.304414 9182 raft_consensus.cc:3055] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:30.304831 9182 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.305615 9182 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.306456 9597 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:30.307026 9182 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:30.307276 9597 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:30.308072 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 197240us
I20260616 19:58:30.308320 9563 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:30.308615 7962 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:30.308893 8011 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.310566 9588 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:30.310931 8011 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.312335 9563 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:30.312073 8011 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.285562 9395 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.013s
I20260616 19:58:30.327188 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:30.327834 7965 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:30.331035 7963 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:30.333423 7963 pending_rounds.cc:85] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 2
I20260616 19:58:30.333700 7963 pending_rounds.cc:107] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260616 19:58:30.333981 7963 raft_consensus.cc:2889] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:30.336117 7961 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:30.338423 8011 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:30.341543 9603 log.cc:927] Time spent T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260616 19:58:30.342098 9480 consensus_queue.cc:237] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.344466 9563 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:30.353408 9515 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:30.365038 9454 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.344568 9377 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260616 19:58:30.367182 7965 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:30.367847 9454 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.339890 9597 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.367929 9462 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.368901 7935 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:30.370416 7935 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:30.370936 9462 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.371405 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:30.372568 9455 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:30.373184 9455 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:30.377545 9182 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:30.378443 9531 raft_consensus.cc:3055] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:30.378852 9531 raft_consensus.cc:740] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.380409 8084 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:30.383261 9293 raft_consensus.cc:993] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:30.389353 9293 raft_consensus.cc:1081] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:30.389706 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:30.390484 8045 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:30.391217 9293 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:58:30.393945 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:30.395479 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.354385 8011 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.397169 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.398386 9515 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:30.401233 8011 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.379624 9531 consensus_queue.cc:260] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.386891 8036 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:30.404714 9597 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:30.406980 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.410333 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:30.412985 9515 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:30.418744 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:30.418865 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 47525.8us
I20260616 19:58:30.422930 8011 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:30.424445 7864 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 4 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:30.426604 7864 catalog_manager.cc:5696] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:30.436094 8011 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.437305 8011 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.445979 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:30.446733 8011 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.447387 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:30.448423 8038 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:30.451249 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:30.451751 8038 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:30.454300 8043 raft_consensus.cc:2393] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:30.474237 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:30.474999 8042 raft_consensus.cc:2393] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:30.448796 7868 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 130) took 7814 ms. Trace:
I20260616 19:58:30.466940 8045 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:30.487664 8038 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
W20260616 19:58:30.494769 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 48101.7us
I20260616 19:58:30.495991 8011 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.503355 8084 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:30.448081 9515 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:30.466995 9377 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:30.508533 8036 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:30.466787 9588 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260616 19:58:30.476002 7868 rpcz_store.cc:276] 0616 19:58:22.634284 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:22.634626 (+ 342us) service_pool.cc:224] Handling call
0616 19:58:30.448740 (+7814114us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:29.284429 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:29.300826 (+ 16397us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:29.300860 (+ 34us) write_op.cc:435] Acquired schema lock
0616 19:58:29.300879 (+ 19us) tablet.cc:662] Decoding operations
0616 19:58:29.301436 (+ 557us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:29.308478 (+ 7042us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:29.315036 (+ 6558us) tablet.cc:685] Acquiring locks for 4 operations
0616 19:58:29.334452 (+ 19416us) tablet.cc:701] Row locks acquired
0616 19:58:29.334470 (+ 18us) write_op.cc:260] PREPARE: finished
0616 19:58:29.334598 (+ 128us) write_op.cc:270] Start()
0616 19:58:29.334722 (+ 124us) write_op.cc:276] Timestamp: P: 1781639909334559 usec, L: 0
0616 19:58:29.334740 (+ 18us) op_driver.cc:348] REPLICATION: starting
0616 19:58:29.378658 (+ 43918us) log.cc:844] Serialized 1705 byte log entry
0616 19:58:29.943097 (+564439us) op_driver.cc:464] REPLICATION: finished
0616 19:58:30.012144 (+ 69047us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e1208 00007fe872d1d401 00007fe872d1cbd8 00007fe872d21871 00007fe872d21829 00007fe872d217b9 00007fe872d21781 00007fe872d209ad 0000000000389354 0000000000389199
0616 19:58:30.064980 (+ 52836us) write_op.cc:301] APPLY: starting
0616 19:58:30.065076 (+ 96us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:30.065225 (+ 149us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:30.065246 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:30.120660 (+ 55414us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:30.125451 (+ 4791us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0616 19:58:30.136804 (+ 11353us) write_op.cc:312] APPLY: finished
0616 19:58:30.174561 (+ 37757us) log.cc:844] Serialized 40 byte log entry
0616 19:58:30.182969 (+ 8408us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:30.183109 (+ 140us) write_op.cc:454] Released schema lock
0616 19:58:30.183156 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:30.183527 (+ 371us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:30.183562 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":121603,"apply.run_cpu_time_us":3275,"apply.run_wall_time_us":118504,"num_ops":4,"prepare.queue_time_us":295368,"prepare.run_cpu_time_us":57628,"prepare.run_wall_time_us":761580,"raft.queue_time_us":136415,"raft.run_cpu_time_us":1201,"raft.run_wall_time_us":139729,"replication_time_us":600373,"spinlock_wait_cycles":99456768,"thread_start_us":445061,"threads_started":4,"wal-append.queue_time_us":272394}]]}
I20260616 19:58:30.515642 9588 consensus_queue.cc:1243] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260616 19:58:30.507632 9455 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:58:30.520539 9377 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:30.521005 9377 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:30.524577 9346 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.013s
I20260616 19:58:30.525182 9481 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:30.526486 9481 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:30.526933 9481 raft_consensus.cc:697] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.527341 9346 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:30.537604 8011 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:30.542210 7935 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:30.528343 8044 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:30.553335 8085 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:30.505687 9585 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:30.561101 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.560539 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.563010 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:30.563601 8112 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:30.564934 7968 pending_rounds.cc:85] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:30.565251 7968 pending_rounds.cc:107] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260616 19:58:30.565470 7968 raft_consensus.cc:2889] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:30.566234 7965 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:30.567530 8119 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:30.567463 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:30.564965 8110 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:30.531183 7865 catalog_manager.cc:5696] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:30.572014 7865 catalog_manager.cc:5696] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:30.606139 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:30.537340 9568 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:30.570261 9572 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
W20260616 19:58:30.609489 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 53775.7us
I20260616 19:58:30.586508 8112 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:30.607012 7865 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 3, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:30.524611 8819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096"
dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:52458
I20260616 19:58:30.617592 7865 catalog_manager.cc:5696] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:30.621987 7865 catalog_manager.cc:5696] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:30.607491 9574 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:30.570745 9221 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:30.571103 9454 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:30.576542 9531 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:30.607861 9395 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:30.623953 7865 catalog_manager.cc:5696] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:30.570456 9589 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
W20260616 19:58:30.625829 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 53720.2us
I20260616 19:58:30.607403 8041 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:30.630427 8819 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:30.638135 8118 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:30.527767 9481 consensus_queue.cc:237] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.639662 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:30.617669 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 26) took 1576 ms. Trace:
I20260616 19:58:30.665284 8112 raft_consensus.cc:606] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership
I20260616 19:58:30.662493 7865 catalog_manager.cc:5696] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:30.618688 9384 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260616 19:58:30.673202 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 63079.6us
W20260616 19:58:30.678637 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 46240.7us
I20260616 19:58:30.681306 8011 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.682428 8011 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.684306 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:30.679334 7865 catalog_manager.cc:5696] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:30.665550 8819 raft_consensus.cc:3060] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:30.686209 7865 catalog_manager.cc:5696] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:30.691334 7865 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:30.693010 7865 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:30.693692 9563 consensus_queue.cc:1059] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:30.694490 9563 consensus_queue.cc:1243] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260616 19:58:30.695130 9563 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:30.694523 7865 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:30.660185 9455 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:30.695874 9563 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:58:30.727572 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 45191.3us
W20260616 19:58:30.728137 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 43914.5us
I20260616 19:58:30.665577 9574 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:30.677956 9454 raft_consensus.cc:697] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:30.700651 9455 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:30.634016 9221 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.653929 9395 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.735713 9597 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:30.729826 9560 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260616 19:58:30.748660 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 67225.6us
I20260616 19:58:30.753280 9221 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.756202 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:30.726933 9572 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:30.751534 8819 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:30.736002 9563 raft_consensus.cc:3055] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:30.756881 8040 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:30.769189 7934 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:30.769186 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:30.769738 9563 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:30.771492 9480 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260616 19:58:30.796073 9563 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.797398 9563 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:30.797863 9454 consensus_queue.cc:237] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.798573 9574 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:30.812618 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 43538.9us
W20260616 19:58:30.851958 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 108377us
I20260616 19:58:30.805078 8819 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:30.754438 9462 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260616 19:58:30.803788 9377 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.867012 9384 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:30.877483 9480 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260616 19:58:30.906471 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 137341us
W20260616 19:58:30.894815 9620 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.089s user 0.001s sys 0.000s
I20260616 19:58:30.763895 9572 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:30.919958 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:58:30.920332 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 146882us
I20260616 19:58:30.920631 7970 raft_consensus.cc:3060] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:30.909857 9480 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:30.918311 9384 raft_consensus.cc:697] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:30.931867 9480 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:30.667940 7864 rpcz_store.cc:276] 0616 19:58:29.041061 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:29.041680 (+ 619us) service_pool.cc:224] Handling call
0616 19:58:30.615363 (+1573683us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:30.429658 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:30.430196 (+ 538us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:30.430231 (+ 35us) write_op.cc:435] Acquired schema lock
0616 19:58:30.430246 (+ 15us) tablet.cc:662] Decoding operations
0616 19:58:30.430648 (+ 402us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:30.430723 (+ 75us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:30.430750 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0616 19:58:30.431059 (+ 309us) tablet.cc:701] Row locks acquired
0616 19:58:30.431078 (+ 19us) write_op.cc:260] PREPARE: finished
0616 19:58:30.431216 (+ 138us) write_op.cc:270] Start()
0616 19:58:30.431324 (+ 108us) write_op.cc:276] Timestamp: P: 1781639910431171 usec, L: 0
0616 19:58:30.431342 (+ 18us) op_driver.cc:348] REPLICATION: starting
0616 19:58:30.432776 (+ 1434us) log.cc:844] Serialized 981 byte log entry
0616 19:58:30.505381 (+ 72605us) op_driver.cc:464] REPLICATION: finished
0616 19:58:30.516415 (+ 11034us) write_op.cc:301] APPLY: starting
0616 19:58:30.516503 (+ 88us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:30.516613 (+ 110us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:30.516706 (+ 93us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:30.517041 (+ 335us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:30.517188 (+ 147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0616 19:58:30.517231 (+ 43us) write_op.cc:312] APPLY: finished
0616 19:58:30.517422 (+ 191us) log.cc:844] Serialized 28 byte log entry
0616 19:58:30.517575 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:30.517659 (+ 84us) write_op.cc:454] Released schema lock
0616 19:58:30.517696 (+ 37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":97584,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":1910,"apply.run_cpu_time_us":1587,"apply.run_wall_time_us":1588,"num_ops":2,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":2599,"prepare.run_wall_time_us":3393,"replication_time_us":42351}]]}
W20260616 19:58:31.187538 9568 consensus_meta.cc:287] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.572s user 0.007s sys 0.020s
I20260616 19:58:31.187402 7970 raft_consensus.cc:2468] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 2.
I20260616 19:58:31.189878 9626 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:31.189056 9455 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.252543 9568 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:31.252876 9626 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:31.254709 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 460619us
I20260616 19:58:31.255631 9626 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:31.043285 9620 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.109s user 0.001s sys 0.000s
W20260616 19:58:31.259274 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 337912us
I20260616 19:58:31.259663 9455 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:31.189915 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:31.264478 9395 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:58:31.271536 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.272359 8110 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:31.198553 9597 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:31.213680 9480 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.241402 9550 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:31.242345 9583 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:31.242836 9570 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.277740 9597 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.278004 9570 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.277747 9583 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.280409 9550 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.280743 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:31.281414 8036 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:31.243067 9221 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.282390 7970 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:58:31.280028 9597 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.282871 9221 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.283828 8036 raft_consensus.cc:1240] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:31.243652 7962 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:31.285197 9480 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.285588 7962 raft_consensus.cc:606] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:31.291765 8819 raft_consensus.cc:1240] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:31.292675 9221 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.280028 9570 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.295260 9531 consensus_queue.cc:1059] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:31.296397 9531 raft_consensus.cc:3055] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:31.296785 9531 raft_consensus.cc:740] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:31.297412 9293 raft_consensus.cc:993] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:31.245970 9377 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.247567 9384 consensus_queue.cc:237] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:31.012516 9623 log.cc:927] Time spent T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.144s user 0.000s sys 0.001s
I20260616 19:58:31.277173 7963 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:31.307912 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:31.307363 7963 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:31.313768 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f03211770a74604a786608de5062cc3"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:31.314558 8115 raft_consensus.cc:606] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:31.314983 8115 raft_consensus.cc:614] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:58:31.315400 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:31.316138 7963 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:31.317238 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:31.318185 7962 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:31.319203 9377 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:31.324903 8011 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:31.282292 9583 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.282503 9550 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.330264 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:31.332939 8786 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:58:31.321256 9477 raft_consensus.cc:1081] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:31.353762 8113 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:31.355084 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:31.339685 9377 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:31.362250 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.362852 8113 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:31.357730 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.363806 8117 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:31.364957 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.365772 8117 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:31.366139 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:31.366554 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:31.367081 8113 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:31.368901 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.369518 8117 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:31.370402 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:31.370728 8036 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260616 19:58:31.371107 8036 pending_rounds.cc:85] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 2
I20260616 19:58:31.371382 8036 pending_rounds.cc:107] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260616 19:58:31.371565 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:31.372516 8041 raft_consensus.cc:2393] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:31.371589 8036 raft_consensus.cc:2889] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:31.387538 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:31.388500 8040 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:31.400925 8115 raft_consensus.cc:606] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership
I20260616 19:58:31.363405 9377 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:31.328437 9531 consensus_queue.cc:260] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:31.409197 8040 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:31.410524 8113 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:31.416828 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:31.418083 8040 raft_consensus.cc:2393] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:31.410934 9377 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:58:31.410337 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 77455.6us
I20260616 19:58:31.423324 8044 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:31.419436 9531 raft_consensus.cc:3060] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:31.427524 8819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e"
dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:52458
I20260616 19:58:31.427228 9649 raft_consensus.cc:2804] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:31.432771 8819 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:31.435814 8819 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:31.435055 9377 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.449455 9377 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.452716 9649 raft_consensus.cc:697] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:58:31.453887 9649 consensus_queue.cc:237] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:31.507185 8084 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:31.451108 8819 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:31.540838 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.542150 8118 raft_consensus.cc:2393] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:31.514366 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:31.518640 8110 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:31.518708 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:31.527680 8112 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:31.535743 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:31.529418 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:31.530834 8119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8890d947e59455187876dccb01c8c06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:31.548200 8119 raft_consensus.cc:606] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:31.548494 8045 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:31.548684 8119 raft_consensus.cc:614] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:58:31.552129 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.555542 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:31.558238 8819 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.560768 8045 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:31.564404 7934 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:31.566103 7934 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:31.569876 9635 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:31.570472 9635 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:31.570946 9635 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:31.570864 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 64117.3us
I20260616 19:58:31.571755 9462 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:58:31.572405 9462 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.572933 9462 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:31.569433 9550 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:31.582577 9531 raft_consensus.cc:2764] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260616 19:58:31.583129 9402 consensus_queue.cc:1059] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260616 19:58:31.589952 9402 raft_consensus.cc:3055] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
W20260616 19:58:31.595050 9641 log.cc:927] Time spent T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.248s user 0.003s sys 0.000s
I20260616 19:58:31.599571 9529 raft_consensus.cc:993] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:31.600342 9529 raft_consensus.cc:1081] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:31.600378 9583 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.601241 9583 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.599331 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:31.602906 8116 raft_consensus.cc:3055] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:31.603363 8116 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:31.604733 8116 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:31.605978 9583 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.607545 8116 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:31.604748 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:31.609210 8115 raft_consensus.cc:606] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:31.609609 8115 raft_consensus.cc:614] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:58:31.610898 7864 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 2 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:31.613178 9628 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.423s user 0.003s sys 0.000s
I20260616 19:58:31.612947 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:31.614032 8112 raft_consensus.cc:2393] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:31.616129 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260616 19:58:31.622578 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:31.605393 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:31.629287 9629 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260616 19:58:31.609755 9640 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.263s user 0.000s sys 0.002s
I20260616 19:58:31.641777 9563 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.657292 9563 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:31.642649 7963 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:31.605998 9586 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.644088 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 752) took 1079 ms. Trace:
I20260616 19:58:31.644726 9597 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.590812 9402 raft_consensus.cc:740] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:31.650182 8116 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:58:31.639839 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:31.637379 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:31.669112 8045 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:58:31.673579 8044 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:31.674188 9402 consensus_queue.cc:260] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.674858 9597 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.675513 9402 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:31.676847 9586 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.677528 9597 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.677567 7964 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:31.679003 9586 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.682909 9563 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:31.661492 7968 rpcz_store.cc:276] 0616 19:58:30.564308 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:30.564452 (+ 144us) service_pool.cc:224] Handling call
0616 19:58:30.564749 (+ 297us) raft_consensus.cc:1471] Updating replica for 2 ops
0616 19:58:30.573162 (+ 8413us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:58:30.573196 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0616 19:58:30.573750 (+ 554us) log.cc:844] Serialized 52 byte log entry
0616 19:58:30.700280 (+126530us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b180026ef60. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c216587 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe86ee7e5ab 00007fe86ee840c3 00007fe86ee84471 00007fe86f2a0def
0616 19:58:30.755676 (+ 55396us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:58:30.769717 (+ 14041us) log.cc:844] Serialized 14 byte log entry
0616 19:58:30.769905 (+ 188us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:58:30.769978 (+ 73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:58:31.612510 (+842532us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b240015f5a8. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c085e94 00007fe86c083f9b 00007fe86c112d39 00007fe86c1129c5 00007fe86c11200d 00007fe86ee684ae 00007fe86ee7fc2c 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1
0616 19:58:31.643612 (+ 31102us) raft_consensus.cc:1685] finished
0616 19:58:31.643642 (+ 30us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:58:31.643882 (+ 240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36908,"prepare.queue_time_us":15987,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":984,"raft.queue_time_us":2645,"raft.run_cpu_time_us":5448,"raft.run_wall_time_us":67006,"spinlock_wait_cycles":119230592,"thread_start_us":55222,"threads_started":1,"wal-append.queue_time_us":367018}
I20260616 19:58:31.684580 7969 raft_consensus.cc:2393] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:31.686442 8784 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:31.690800 9602 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:31.691434 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:31.691797 9602 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.692523 9649 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:31.693086 9649 raft_consensus.cc:697] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:58:31.694540 9602 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:31.694314 9649 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:31.711874 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 120) took 5549 ms. Trace:
I20260616 19:58:31.697409 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:31.712751 7865 rpcz_store.cc:276] 0616 19:58:26.162142 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:26.162383 (+ 241us) service_pool.cc:224] Handling call
0616 19:58:31.711530 (+5549147us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:30.737250 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:30.738962 (+ 1712us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:30.739013 (+ 51us) write_op.cc:435] Acquired schema lock
0616 19:58:30.739039 (+ 26us) tablet.cc:662] Decoding operations
0616 19:58:30.743289 (+ 4250us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:30.755432 (+ 12143us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:30.757196 (+ 1764us) tablet.cc:685] Acquiring locks for 12 operations
0616 19:58:30.880055 (+122859us) tablet.cc:701] Row locks acquired
0616 19:58:30.880081 (+ 26us) write_op.cc:260] PREPARE: finished
0616 19:58:30.880299 (+ 218us) write_op.cc:270] Start()
0616 19:58:30.880459 (+ 160us) write_op.cc:276] Timestamp: P: 1781639910880236 usec, L: 0
0616 19:58:30.880487 (+ 28us) op_driver.cc:348] REPLICATION: starting
0616 19:58:30.881369 (+ 882us) log.cc:844] Serialized 4809 byte log entry
0616 19:58:30.918506 (+ 37137us) op_driver.cc:464] REPLICATION: finished
0616 19:58:30.928949 (+ 10443us) write_op.cc:301] APPLY: starting
0616 19:58:31.234793 (+305844us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:31.235189 (+ 396us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:31.235416 (+ 227us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:31.237376 (+ 1960us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:31.237714 (+ 338us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0616 19:58:31.237787 (+ 73us) write_op.cc:312] APPLY: finished
0616 19:58:31.238248 (+ 461us) log.cc:844] Serialized 88 byte log entry
0616 19:58:31.317738 (+ 79490us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:31.317875 (+ 137us) write_op.cc:454] Released schema lock
0616 19:58:31.317931 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:31.320743 (+ 2812us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:31.320772 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":32477,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":1894,"apply.run_cpu_time_us":12413,"apply.run_wall_time_us":391737,"num_ops":12,"prepare.queue_time_us":15019,"prepare.run_cpu_time_us":5880,"prepare.run_wall_time_us":169046,"replication_time_us":23177}]]}
I20260616 19:58:31.716717 7962 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
W20260616 19:58:31.719532 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 54320.5us
W20260616 19:58:31.721114 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 72455.9us
I20260616 19:58:31.721695 9454 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260616 19:58:31.679123 7853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 114024us
I20260616 19:58:31.719435 7968 raft_consensus.cc:1240] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:31.723894 9377 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:31.726790 9454 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:31.729909 9377 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.733162 9454 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:31.743510 8117 raft_consensus.cc:2468] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 2.
I20260616 19:58:31.748245 7934 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:31.749099 9377 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.750492 9586 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:31.753019 9221 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:31.753103 7934 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:31.753547 9221 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.753942 9221 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:31.757001 9390 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:31.757654 9390 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.759795 9390 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.757092 9586 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:31.751559 9635 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.784652 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:31.788760 8115 raft_consensus.cc:606] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Received request to transfer leadership
I20260616 19:58:31.789089 8115 raft_consensus.cc:614] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:58:31.785580 8044 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:31.775883 9586 raft_consensus.cc:697] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:31.796926 8040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:31.811553 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.812220 8117 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:31.815196 8011 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:31.816486 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.816903 8009 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:31.821842 9637 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:31.826714 9572 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:31.827095 9637 raft_consensus.cc:697] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:31.828362 9637 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.840413 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:31.846388 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:31.865644 7964 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:31.866561 8038 raft_consensus.cc:1240] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
W20260616 19:58:31.871243 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:31.874361 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 57732us
I20260616 19:58:31.882907 8040 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:31.883518 8040 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:31.887533 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:31.888620 8036 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:31.889384 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:31.876991 9635 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.905360 8009 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:31.908922 7934 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:31.909408 9572 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:31.911266 9455 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:31.911875 9455 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.912297 9455 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:31.916059 7964 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:58:31.918525 9480 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:31.919121 9480 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:31.920516 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:31.930403 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:31.927351 8040 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.874516 9586 consensus_queue.cc:237] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.943631 8038 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:31.948764 9456 consensus_queue.cc:1059] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:31.951722 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:58:31.953159 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 47943.5us
I20260616 19:58:31.953403 9480 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.928757 9637 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:58:31.915665 9563 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260616 19:58:31.916528 9454 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:31.961844 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 50379.3us
I20260616 19:58:31.951927 9402 consensus_queue.cc:1059] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260616 19:58:31.933892 9462 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.981462 9221 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.982399 9402 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:31.982798 9402 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:31.983201 9454 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.983654 9402 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.984422 9384 raft_consensus.cc:993] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:31.984947 9384 raft_consensus.cc:1081] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:31.985091 9402 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:58:31.987150 9663 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.105s user 0.001s sys 0.000s
W20260616 19:58:31.989143 9395 consensus_queue.cc:1175] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" responder_term: 6 status { last_received { term: 6 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260616 19:58:31.990597 9294 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:31.991242 9294 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.971202 9377 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:31.992822 9377 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:31.993348 7968 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:31.994971 9377 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:31.978477 9529 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260616 19:58:31.976421 9480 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:31.980203 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:31.980605 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.002218 7964 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260616 19:58:32.003453 9597 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:32.003965 9597 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.006121 9597 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.006623 9455 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.008736 9649 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:32.010370 9480 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.012887 9480 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.013640 8110 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:32.025969 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.026657 8112 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:32.027477 9294 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:31.983670 9221 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.034353 8040 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.038219 9549 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:32.038691 9549 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:32.045048 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.053028 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.064687 9455 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.066264 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:32.069332 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.046641 9637 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:32.076845 7968 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:32.076769 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.078060 8110 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:32.079823 7934 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:32.083765 7970 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:32.084224 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.082525 9637 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.086110 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:32.086745 8036 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:32.086953 9637 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:32.088059 9665 log.cc:927] Time spent T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260616 19:58:32.039603 9549 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.090322 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:32.091024 7969 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:32.091096 9549 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:32.091842 9462 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.067865 9456 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260616 19:58:32.100749 8119 raft_consensus.cc:2410] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260616 19:58:32.101676 9661 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.215s user 0.001s sys 0.000s
I20260616 19:58:32.094437 9294 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:32.097898 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:32.105427 7969 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:32.107229 8011 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:32.108991 8045 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:32.110167 8112 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:32.111754 9455 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:32.113668 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.114265 8040 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:32.112984 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:58:32.110005 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 73402.2us
I20260616 19:58:32.112423 8116 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:32.117986 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:32.122973 8044 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:32.125111 9673 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:32.125558 9673 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:32.125900 9673 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:32.127900 8117 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:32.129101 8041 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260616 19:58:32.130048 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:32.121706 9597 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260616 19:58:32.131009 8040 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:32.111416 9529 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260616 19:58:32.135859 7934 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:32.137792 9629 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:32.117043 9455 raft_consensus.cc:697] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.111637 9294 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.140880 9455 consensus_queue.cc:237] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.141765 9294 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:32.143014 9462 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:32.143566 9462 raft_consensus.cc:697] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.144558 7934 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:32.121940 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:32.145562 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.146114 9629 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.149129 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:32.149324 9629 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.151321 7969 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:32.153436 9221 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:32.154143 9221 raft_consensus.cc:697] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.156235 7962 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:32.158492 9672 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:32.159016 9672 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.161705 9672 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.161775 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.162433 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.163082 8112 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:32.162572 9462 consensus_queue.cc:237] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.164696 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.165413 8112 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:32.166647 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.169003 8119 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
W20260616 19:58:32.172791 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 52527.3us
I20260616 19:58:32.155270 9221 consensus_queue.cc:237] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.175124 8112 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260616 19:58:32.176375 9672 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:32.176407 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.177768 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.178568 7970 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:32.176930 9672 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:32.184221 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.191747 7934 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:32.193447 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.195837 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.196409 8110 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:32.196662 8110 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:32.198040 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.198902 7969 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:32.195323 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.204108 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.209962 8009 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:32.203423 8110 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.210798 8110 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:32.213169 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:32.213915 8036 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:32.213268 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.214907 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.215510 8044 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:32.217136 7935 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:32.217793 8111 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260616 19:58:32.218153 8111 pending_rounds.cc:85] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 2
I20260616 19:58:32.218401 8111 pending_rounds.cc:107] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260616 19:58:32.218535 7935 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:32.219421 7967 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:32.219523 8117 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:32.199651 9572 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:32.220542 9550 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:32.218679 8111 raft_consensus.cc:2889] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:32.220808 9572 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.222079 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.223063 9572 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.199992 9583 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:32.224668 9583 raft_consensus.cc:697] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.225765 9583 consensus_queue.cc:237] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.233546 9669 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:32.234670 8009 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:32.236145 8042 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260616 19:58:32.225301 9672 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.237320 8043 raft_consensus.cc:2393] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:32.237497 8009 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:32.240480 8040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:58:32.241117 8040 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:32.241510 8040 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:32.242165 7969 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:32.233568 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.246470 9597 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:32.247189 9597 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:58:32.247584 9597 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:32.247937 9597 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:32.223868 9673 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.253145 8045 raft_consensus.cc:2368] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6.
I20260616 19:58:32.259475 8110 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:32.261284 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:32.265334 7935 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:32.267976 8040 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.269085 9572 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:32.270205 9377 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:32.274353 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.277932 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:32.280597 9377 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:32.247974 9672 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:32.289345 8040 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.283689 9480 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:32.283214 9462 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:58:32.270900 9586 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260616 19:58:32.288947 9673 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.305291 7963 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260616 19:58:32.306911 9454 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:32.284567 9563 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260616 19:58:32.330435 9654 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:32.290860 9635 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260616 19:58:32.331660 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:32.303124 9462 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:32.320389 9563 consensus_queue.cc:1243] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260616 19:58:32.351431 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 48993.4us
W20260616 19:58:32.422401 9678 log.cc:927] Time spent T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.151s user 0.001s sys 0.000s
I20260616 19:58:32.425913 7868 catalog_manager.cc:5696] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:32.428119 7868 catalog_manager.cc:5696] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.351974 9462 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:32.435292 7868 catalog_manager.cc:5696] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:32.440418 7935 leader_election.cc:400] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6.
I20260616 19:58:32.442653 7868 catalog_manager.cc:5696] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.453404 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:32.455282 9480 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:32.455598 8038 raft_consensus.cc:1275] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:32.455859 9480 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:32.457072 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 185931us
I20260616 19:58:32.456966 9673 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260616 19:58:32.460897 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 45554.5us
W20260616 19:58:32.461220 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:32.469903 7868 catalog_manager.cc:5696] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:32.471761 7868 catalog_manager.cc:5696] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.474016 9654 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:32.473495 7868 catalog_manager.cc:5696] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.479452 7868 catalog_manager.cc:5696] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 2 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.481410 7868 catalog_manager.cc:5696] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 5 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.436295 9597 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.486521 9395 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:32.487255 9597 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.484536 8085 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:32.491068 9529 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:32.493320 8085 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:32.490046 9583 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260616 19:58:32.491518 9529 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:32.501760 8042 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:32.512568 8112 raft_consensus.cc:1275] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:32.517709 9676 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260616 19:58:32.518993 9480 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.508795 9529 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:32.523799 9675 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:58:32.524304 9675 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:32.524744 9675 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:32.527734 9462 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.530414 9462 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:32.531708 9683 log.cc:927] Time spent T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.108s user 0.000s sys 0.003s
I20260616 19:58:32.542820 9586 consensus_queue.cc:1059] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:32.544008 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.544622 8040 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:32.545003 9586 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:32.545558 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.546931 9550 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:32.547358 9673 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:32.547923 9673 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.545373 9586 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.549998 9673 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.551813 8009 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:32.550873 9686 log.cc:927] Time spent T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.122s user 0.001s sys 0.000s
I20260616 19:58:32.550684 9586 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.556962 9586 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:58:32.557363 9682 log.cc:927] Time spent T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.098s user 0.000s sys 0.002s
I20260616 19:58:32.572636 8118 raft_consensus.cc:1275] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260616 19:58:32.575802 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 294499us
I20260616 19:58:32.573119 7868 catalog_manager.cc:5696] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.583416 7868 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.582203 8011 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:32.591712 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.591804 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.599097 8040 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:32.592574 8112 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:32.580093 9673 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:32.611984 7868 catalog_manager.cc:5696] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:32.620208 7868 catalog_manager.cc:5696] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
W20260616 19:58:32.629876 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 45869.4us
I20260616 19:58:32.630035 7868 catalog_manager.cc:5696] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.632300 7935 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:32.587929 9637 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260616 19:58:32.633533 8009 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:32.631999 7868 catalog_manager.cc:5696] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 4, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:32.645504 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.646173 8114 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:32.647886 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.648679 8117 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:58:32.653934 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 71819.5us
I20260616 19:58:32.636284 9572 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:32.659091 9563 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:58:32.682233 7961 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260616 19:58:32.682720 9670 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260616 19:58:32.691552 9629 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:32.692139 9629 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.693163 9629 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.699234 8117 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:32.686482 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.701020 8040 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:32.656461 9462 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260616 19:58:32.720844 7961 raft_consensus.cc:1240] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:32.722664 8084 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:32.690939 9563 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:32.719983 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 63209us
I20260616 19:58:32.730800 9480 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.728457 9563 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:32.752429 8040 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:32.725216 9529 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.778162 9704 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:32.781976 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 70570us
I20260616 19:58:32.754254 9676 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:32.799453 8116 raft_consensus.cc:2368] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5.
W20260616 19:58:32.799291 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 76700.9us
I20260616 19:58:32.809371 8114 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:32.797631 9676 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:58:32.827797 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 127137us
I20260616 19:58:32.832584 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:32.834460 8043 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:32.841758 7964 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:32.842123 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:32.815083 9676 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:32.846346 8043 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:32.808753 9704 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:32.858067 7916 auto_rebalancer.cc:855] Removing replace marker failed: Replica 6827184e3cef42c2ada15b2d31afee4b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:32.858003 9706 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
W20260616 19:58:32.858776 7916 auto_rebalancer.cc:863] Could not move replica: Illegal state: not currently leader
W20260616 19:58:32.859124 7916 auto_rebalancer.cc:301] scheduled replica move failed to complete: Illegal state: not currently leader
I20260616 19:58:32.862186 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.862946 8110 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:58:32.865928 9675 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.867254 8115 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:58:32.868319 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:32.875285 9377 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.875780 9529 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:32.885110 9675 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:32.887391 9709 consensus_queue.cc:1059] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:32.889456 8043 raft_consensus.cc:1240] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:58:32.888233 9706 consensus_queue.cc:1059] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:32.890810 9710 raft_consensus.cc:3055] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:32.891291 9706 raft_consensus.cc:3055] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:32.891450 9710 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:32.891676 9706 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:32.892465 9710 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.892854 9706 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.893760 9710 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:32.894325 9706 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:32.902583 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:32.903435 7966 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:32.903685 8115 raft_consensus.cc:1275] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260616 19:58:32.843312 9563 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.909914 8110 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:32.911525 9563 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:32.911787 8119 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:32.913681 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:32.914490 7966 raft_consensus.cc:2393] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:32.921883 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.922838 8117 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260616 19:58:32.924042 7961 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:32.926774 7935 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:32.936565 7963 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:32.938720 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.941663 8042 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:32.944417 8114 raft_consensus.cc:1275] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:32.944882 9654 consensus_queue.cc:1048] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:32.945415 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:32.948184 8045 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260616 19:58:32.950481 9384 consensus_queue.cc:1059] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:32.954128 7969 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:32.955128 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:32.956995 8009 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:32.957542 8117 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260616 19:58:32.961728 9384 raft_consensus.cc:3055] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:32.962365 9384 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:32.964504 9707 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:32.965147 9707 raft_consensus.cc:697] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:32.966133 9707 consensus_queue.cc:237] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.967092 9377 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:32.968724 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:32.969333 8111 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:32.971757 9669 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:32.973946 9669 consensus_queue.cc:1048] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:32.969889 8042 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:32.934867 9704 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:32.983299 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 56617.3us
I20260616 19:58:32.963425 9384 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:32.984694 9586 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:32.993539 8009 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:33.001628 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:32.997673 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:32.997217 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:33.016057 7963 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:33.016142 7961 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:33.020606 8009 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:33.025209 8009 leader_election.cc:400] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5.
I20260616 19:58:33.025499 8009 leader_election.cc:403] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260616 19:58:33.028756 8117 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:33.034102 8043 raft_consensus.cc:1275] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:33.021052 9384 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:33.045349 8111 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:33.049844 8114 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260616 19:58:33.021641 9586 raft_consensus.cc:697] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:33.054931 7961 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:33.024188 9704 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:32.993680 9529 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260616 19:58:33.034015 9572 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:58:33.064783 8036 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:33.033931 9688 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5.
I20260616 19:58:33.073882 8111 raft_consensus.cc:1275] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260616 19:58:33.077572 9696 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.344s user 0.000s sys 0.008s
I20260616 19:58:33.080107 8041 raft_consensus.cc:1275] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:33.082641 9719 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:33.061316 9672 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260616 19:58:33.033735 9705 consensus_queue.cc:1059] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260616 19:58:33.066138 9572 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:33.100247 9694 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:33.100732 9572 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:33.106870 7963 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:33.110087 9586 consensus_queue.cc:237] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.114969 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:33.119498 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:58:33.147272 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 89729.6us
W20260616 19:58:33.186937 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 137158us
I20260616 19:58:33.087754 9676 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.193397 8042 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:33.194892 8011 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:33.196074 9672 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:33.207566 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 158979us
I20260616 19:58:33.210673 8009 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:33.214352 9729 log.cc:927] Time spent T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.095s user 0.000s sys 0.001s
I20260616 19:58:33.214564 8113 raft_consensus.cc:1240] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:58:33.215831 9705 raft_consensus.cc:3055] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:33.142834 9480 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.055s
I20260616 19:58:33.216966 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:33.221112 8038 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:33.252321 8085 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:33.188216 9629 consensus_queue.cc:1059] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260616 19:58:33.216300 9705 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:58:33.298707 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 46458.6us
W20260616 19:58:33.298604 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 76182.7us
I20260616 19:58:33.228606 9395 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:33.197880 9692 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260616 19:58:33.207904 9704 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:33.273105 9377 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
W20260616 19:58:33.297781 9729 log.cc:927] Time spent T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260616 19:58:33.227712 9688 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260616 19:58:33.301254 9731 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:33.320909 9732 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:33.307096 9669 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:33.340482 9377 raft_consensus.cc:697] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:33.343633 9602 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:33.378830 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 75021.4us
I20260616 19:58:33.387290 8116 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:33.343416 9654 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260616 19:58:33.380714 9731 raft_consensus.cc:697] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:33.394881 9731 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:33.396379 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:33.389174 9735 log.cc:927] Time spent T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
I20260616 19:58:33.360891 9669 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.399914 9704 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.353699 9732 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.413329 9377 consensus_queue.cc:237] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.420626 9676 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:33.351390 9705 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.423040 9705 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:33.429769 9563 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:33.430498 9563 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.381007 9602 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.523092 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:33.462478 9395 raft_consensus.cc:697] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:33.527768 9572 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.649871 9384 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:33.680923 9563 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.695379 9602 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:33.727604 9695 raft_consensus.cc:3055] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:33.728127 9695 raft_consensus.cc:740] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:33.729075 9695 consensus_queue.cc:260] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.730412 9695 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:33.690178 9384 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.741140 9384 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.742492 9395 consensus_queue.cc:237] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.747215 9563 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:33.747821 9563 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.748797 9739 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:33.749351 9739 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.750555 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:33.751317 8038 raft_consensus.cc:2393] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:58:33.751600 9739 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.752763 9572 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.755848 9563 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.802965 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:33.824687 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:33.825333 8040 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:33.841675 8040 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:33.850384 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:33.851853 9454 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:33.852584 9454 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:33.859396 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:33.860215 9669 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.861481 9454 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.865873 9695 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260616 19:58:33.881973 9728 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.582s user 0.000s sys 0.009s
I20260616 19:58:33.876466 9732 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:33.874035 9673 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:33.885123 9673 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.887952 9673 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:33.888720 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 60729.1us
I20260616 19:58:33.894235 9690 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:33.904405 8084 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:33.891487 9758 log.cc:927] Time spent T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.057s user 0.001s sys 0.000s
I20260616 19:58:33.907008 9742 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:58:33.910034 8114 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:33.910468 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:33.911116 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:33.911880 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:33.912689 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:33.912783 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:33.913333 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:33.917374 8084 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:33.918216 9531 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:33.918371 8119 raft_consensus.cc:1275] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:58:33.918650 9531 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:33.919044 9531 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:33.920682 8110 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:58:33.933249 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:33.929741 9531 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:33.933966 7969 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:33.935405 9531 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:33.941947 7864 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 6 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:33.944494 7864 catalog_manager.cc:5696] T 2bf8afc79e3c417c814ac2fb57557096 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 2 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:33.952527 9707 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:33.957108 7961 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:33.958173 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:33.958753 7967 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 5
I20260616 19:58:33.968478 7967 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
W20260616 19:58:33.970438 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 43546.5us
I20260616 19:58:34.019570 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:34.020423 8111 raft_consensus.cc:2368] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:34.023272 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:34.026568 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:34.031299 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 48796.6us
I20260616 19:58:34.032433 8009 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:34.033403 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:34.033209 9673 consensus_queue.cc:1048] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:34.034082 9732 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:34.034626 9732 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:34.035576 9732 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.041541 8111 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:34.041852 8085 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:34.041736 9759 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:34.042732 9715 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:34.043254 9715 raft_consensus.cc:697] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:34.044116 9715 consensus_queue.cc:237] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.046155 8111 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:34.047966 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:34.049825 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:34.050511 8039 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:34.058480 9635 consensus_queue.cc:1059] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:34.061277 8039 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:34.066483 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:34.067813 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:34.071434 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:34.072232 7970 raft_consensus.cc:2393] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:58:34.078840 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:34.079437 7970 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260616 19:58:34.079798 7970 pending_rounds.cc:85] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 2
I20260616 19:58:34.080018 7970 pending_rounds.cc:107] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260616 19:58:34.080307 7970 raft_consensus.cc:2889] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:34.080901 7961 raft_consensus.cc:1275] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260616 19:58:34.081408 7961 pending_rounds.cc:85] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7: Aborting all ops after (but not including) 1
I20260616 19:58:34.081744 7961 pending_rounds.cc:107] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260616 19:58:34.082067 7961 raft_consensus.cc:2889] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:34.090384 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 28) took 1354 ms. Trace:
I20260616 19:58:34.090837 7864 rpcz_store.cc:276] 0616 19:58:32.753995 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:32.760667 (+ 6672us) service_pool.cc:224] Handling call
0616 19:58:34.090277 (+1329610us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:33.962557 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:33.962966 (+ 409us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:33.962990 (+ 24us) write_op.cc:435] Acquired schema lock
0616 19:58:33.963002 (+ 12us) tablet.cc:662] Decoding operations
0616 19:58:33.963311 (+ 309us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:33.963374 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:33.963392 (+ 18us) tablet.cc:685] Acquiring locks for 2 operations
0616 19:58:33.963635 (+ 243us) tablet.cc:701] Row locks acquired
0616 19:58:33.963648 (+ 13us) write_op.cc:260] PREPARE: finished
0616 19:58:33.963756 (+ 108us) write_op.cc:270] Start()
0616 19:58:33.963843 (+ 87us) write_op.cc:276] Timestamp: P: 1781639913963724 usec, L: 0
0616 19:58:33.963857 (+ 14us) op_driver.cc:348] REPLICATION: starting
0616 19:58:33.970524 (+ 6667us) log.cc:844] Serialized 981 byte log entry
0616 19:58:33.986323 (+ 15799us) op_driver.cc:464] REPLICATION: finished
0616 19:58:33.986690 (+ 367us) write_op.cc:301] APPLY: starting
0616 19:58:33.986788 (+ 98us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:33.986919 (+ 131us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:33.987027 (+ 108us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:33.987432 (+ 405us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:33.987637 (+ 205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0616 19:58:33.987698 (+ 61us) write_op.cc:312] APPLY: finished
0616 19:58:33.987908 (+ 210us) log.cc:844] Serialized 28 byte log entry
0616 19:58:33.988073 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:33.988189 (+ 116us) write_op.cc:454] Released schema lock
0616 19:58:33.988228 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:33.988662 (+ 434us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:33.988696 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":104308,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":1898,"apply.run_wall_time_us":1918,"num_ops":2,"prepare.queue_time_us":11663,"prepare.run_cpu_time_us":2479,"prepare.run_wall_time_us":23423,"replication_time_us":22320,"thread_start_us":11329,"threads_started":1}]]}
I20260616 19:58:34.101145 8111 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:34.119863 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:34.121490 7967 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:34.104244 9689 raft_consensus.cc:3055] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:34.132953 8011 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:34.134208 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:34.134764 9673 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:34.135426 9673 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:34.139519 7868 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55744 (request call id 129) took 3497 ms. Trace:
I20260616 19:58:34.135205 7866 catalog_manager.cc:5696] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:34.139946 7868 rpcz_store.cc:276] 0616 19:58:30.678383 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:30.682031 (+ 3648us) service_pool.cc:224] Handling call
0616 19:58:34.139412 (+3457381us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:32.847155 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:32.847602 (+ 447us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:32.847625 (+ 23us) write_op.cc:435] Acquired schema lock
0616 19:58:32.847636 (+ 11us) tablet.cc:662] Decoding operations
0616 19:58:32.874090 (+ 26454us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:32.874166 (+ 76us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:32.874194 (+ 28us) tablet.cc:685] Acquiring locks for 15 operations
0616 19:58:32.875302 (+ 1108us) tablet.cc:701] Row locks acquired
0616 19:58:32.875324 (+ 22us) write_op.cc:260] PREPARE: finished
0616 19:58:32.875478 (+ 154us) write_op.cc:270] Start()
0616 19:58:32.875585 (+ 107us) write_op.cc:276] Timestamp: P: 1781639912875432 usec, L: 0
0616 19:58:32.875605 (+ 20us) op_driver.cc:348] REPLICATION: starting
0616 19:58:32.885739 (+ 10134us) log.cc:844] Serialized 5973 byte log entry
0616 19:58:33.117208 (+231469us) op_driver.cc:464] REPLICATION: finished
0616 19:58:33.447759 (+330551us) write_op.cc:301] APPLY: starting
0616 19:58:33.462692 (+ 14933us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:33.600996 (+138304us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:33.683685 (+ 82689us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:33.716224 (+ 32539us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:33.716633 (+ 409us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0616 19:58:33.716710 (+ 77us) write_op.cc:312] APPLY: finished
0616 19:58:33.717238 (+ 528us) log.cc:844] Serialized 106 byte log entry
0616 19:58:33.718715 (+ 1477us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:33.718957 (+ 242us) write_op.cc:454] Released schema lock
0616 19:58:33.719042 (+ 85us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:33.732571 (+ 13529us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:33.732605 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26933,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":264106,"apply.run_cpu_time_us":6138,"apply.run_wall_time_us":292240,"num_ops":15,"prepare.queue_time_us":54373,"prepare.run_cpu_time_us":4421,"prepare.run_wall_time_us":41615,"replication_time_us":241467,"thread_start_us":214256,"threads_started":2}]]}
I20260616 19:58:34.144531 7866 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:34.146831 7866 catalog_manager.cc:5696] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:34.150776 7866 catalog_manager.cc:5696] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:34.135289 8011 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:34.153821 8110 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260616 19:58:34.154220 8110 pending_rounds.cc:85] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 3
I20260616 19:58:34.154423 8110 pending_rounds.cc:107] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260616 19:58:34.154666 8110 raft_consensus.cc:2889] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:34.157719 9732 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:34.158361 8036 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:34.152074 7866 catalog_manager.cc:5696] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 4, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:34.161890 7866 catalog_manager.cc:5696] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:34.172521 8110 raft_consensus.cc:1240] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:58:34.126087 9689 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:34.177271 7866 catalog_manager.cc:5696] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:34.181216 7866 catalog_manager.cc:5696] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:34.198127 9689 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.199389 9689 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:34.188169 7866 catalog_manager.cc:5696] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
W20260616 19:58:34.204751 9768 log.cc:927] Time spent T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.087s user 0.000s sys 0.000s
W20260616 19:58:34.209822 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49126.1us
I20260616 19:58:34.216192 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:34.220552 9732 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:34.206091 7866 catalog_manager.cc:5696] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:34.226019 7866 catalog_manager.cc:5696] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:34.228966 7866 catalog_manager.cc:5696] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:34.231026 7866 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 4, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:34.234747 7866 catalog_manager.cc:5696] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:34.619326 7866 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:34.622749 7866 catalog_manager.cc:5696] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260616 19:58:34.798770 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:34.802664 8116 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s user 0.009s sys 0.007s
W20260616 19:58:34.803109 8116 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s user 0.009s sys 0.007s
W20260616 19:58:34.810460 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 630280us
W20260616 19:58:34.814733 8115 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.651s user 0.006s sys 0.010s
W20260616 19:58:34.815088 8115 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s user 0.006s sys 0.010s
W20260616 19:58:34.844988 8039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s user 0.023s sys 0.009s
W20260616 19:58:34.845399 8039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s user 0.023s sys 0.009s
W20260616 19:58:34.861018 9689 consensus_meta.cc:287] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.661s user 0.009s sys 0.005s
I20260616 19:58:34.900573 9690 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:34.903412 9384 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:34.903980 9384 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.906018 9384 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:34.906271 9692 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:34.906723 9692 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.909698 9692 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:34.901335 9690 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.910917 9384 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260616 19:58:34.917112 7961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s user 0.016s sys 0.003s
I20260616 19:58:34.917439 9669 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:34.917486 7961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s user 0.016s sys 0.003s
W20260616 19:58:34.918035 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:34.918064 9669 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.930442 9669 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:34.948537 7934 leader_election.cc:400] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:34.948954 7934 leader_election.cc:403] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:58:34.943048 9586 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:34.940693 9690 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:34.942267 9669 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:34.956127 9549 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:34.957497 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 146242us
I20260616 19:58:34.957253 9586 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.960232 7934 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:34.966073 9549 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.954296 9692 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5.
I20260616 19:58:34.972338 7934 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:34.975890 9719 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260616 19:58:34.978636 9635 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:34.976006 9676 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:34.981216 9676 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.974392 9695 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:58:34.987519 7970 raft_consensus.cc:1275] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:34.965293 9669 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:34.976220 9759 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:58:34.993762 9462 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:34.989101 9695 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:34.996970 7964 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:34.985066 9602 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.005764 9676 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.007740 9586 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.002748 9741 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.000952 9695 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:35.001258 9694 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:35.024421 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.026175 7970 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:35.034091 8110 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:35.044234 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.000779 9759 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:35.020694 9602 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.023161 9741 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.022986 9694 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.074050 9549 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.074709 9669 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.075897 9759 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:35.076694 9741 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.082212 9602 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.083217 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.104084 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.082515 9695 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.116212 9695 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.118328 9694 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.126463 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.129345 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.130119 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.131107 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.131305 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.131667 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:35.132225 8112 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:35.132236 8114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30950534b1834f6886db15f85f0abf14"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:35.133306 8114 raft_consensus.cc:606] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.133644 8114 raft_consensus.cc:614] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:58:35.134438 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.148211 8112 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:58:35.149480 7934 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:35.152798 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:35.153441 8116 raft_consensus.cc:606] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.156738 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:35.157234 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:35.157953 9720 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:35.159637 9759 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:58:35.160140 9759 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:35.160619 9759 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:35.164377 9395 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:35.166568 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:35.167301 8116 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.169775 9695 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:58:35.173049 8085 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:35.179432 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:35.169425 7964 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:35.180115 8110 raft_consensus.cc:606] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.180485 8110 raft_consensus.cc:614] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260616 19:58:35.182813 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 966694us
I20260616 19:58:35.183790 8011 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.175757 9720 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:35.188325 9705 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:35.191221 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:35.191844 8116 raft_consensus.cc:606] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.192206 8116 raft_consensus.cc:614] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260616 19:58:35.195755 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 167626us
I20260616 19:58:35.173188 9695 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:35.203047 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:35.207239 8036 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:35.210037 8040 raft_consensus.cc:1275] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:58:35.214898 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.215230 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.203344 9739 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:35.237145 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.243254 9695 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.247615 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.248769 8043 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:35.249848 9741 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.250437 9741 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.252602 9741 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.252602 9759 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:35.260455 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 61748.2us
I20260616 19:58:35.212675 9597 consensus_queue.cc:1048] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260616 19:58:35.216466 9673 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260616 19:58:35.269214 8043 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260616 19:58:35.271363 8043 pending_rounds.cc:85] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 3
I20260616 19:58:35.271618 8043 pending_rounds.cc:107] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260616 19:58:35.271602 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.272024 8043 raft_consensus.cc:2889] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:35.272333 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.270641 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:35.278496 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 60423.3us
I20260616 19:58:35.264746 9741 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260616 19:58:35.286194 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:35.297524 9796 log.cc:927] Time spent T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.102s user 0.001s sys 0.000s
I20260616 19:58:35.302642 9705 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:35.303182 9705 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.305166 9705 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.305639 9705 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:35.307322 9759 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.310149 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.312991 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.314127 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.314661 8040 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:35.314931 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.315407 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.316006 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:35.318223 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.318789 9745 raft_consensus.cc:993] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:35.319284 9745 raft_consensus.cc:1081] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:35.324020 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:35.324875 7969 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:58:35.335155 7933 consensus_peers.cc:597] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d19ee585ead4942a40096f7aef3ba50. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:35.337095 9402 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:35.337445 7969 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:35.339051 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:35.342672 9531 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:35.343235 9745 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:35.343456 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:35.343818 9745 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:35.345600 8040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:35.346194 8040 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:35.346613 8040 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:35.351670 8084 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.367007 9740 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:58:35.367502 9740 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.367856 9740 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:35.368857 9531 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260616 19:58:35.371081 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 91636.3us
I20260616 19:58:35.371415 9759 raft_consensus.cc:993] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:35.372042 9759 raft_consensus.cc:1081] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:35.344751 9745 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.377266 8084 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.378793 8040 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:35.379716 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 105439us
I20260616 19:58:35.385105 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:35.386108 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:35.388764 8036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:35.389303 8036 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:35.380870 9602 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:35.389673 8036 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:35.396202 8113 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:35.394797 8009 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.398592 8040 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.413276 7961 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: []
I20260616 19:58:35.412366 9690 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260616 19:58:35.418779 9531 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:35.422644 9740 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.426182 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.428449 8011 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.432001 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.433597 8115 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260616 19:58:35.433712 8043 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:35.419633 9531 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.434300 8114 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:58:35.435075 8119 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:58:35.435629 9384 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:35.441448 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.442824 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:35.445030 8111 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:35.445339 8111 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:35.453262 8009 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.458807 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 121) took 2116 ms. Trace:
I20260616 19:58:35.445640 9597 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:35.461319 8111 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.462579 8111 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:35.477591 8036 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.491904 8041 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260616 19:58:35.497220 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 63992.1us
W20260616 19:58:35.501233 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 59868.8us
I20260616 19:58:35.504639 8111 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:35.507270 8036 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.513186 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:35.513784 8112 raft_consensus.cc:3055] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:35.514076 8112 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:35.459340 7866 rpcz_store.cc:276] 0616 19:58:33.341992 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:33.346135 (+ 4143us) service_pool.cc:224] Handling call
0616 19:58:35.458726 (+2112591us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:34.652539 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:34.653276 (+ 737us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:34.653317 (+ 41us) write_op.cc:435] Acquired schema lock
0616 19:58:34.653335 (+ 18us) tablet.cc:662] Decoding operations
0616 19:58:34.654758 (+ 1423us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:34.654820 (+ 62us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:34.654837 (+ 17us) tablet.cc:685] Acquiring locks for 16 operations
0616 19:58:34.656088 (+ 1251us) tablet.cc:701] Row locks acquired
0616 19:58:34.656111 (+ 23us) write_op.cc:260] PREPARE: finished
0616 19:58:34.656300 (+ 189us) write_op.cc:270] Start()
0616 19:58:34.657987 (+ 1687us) write_op.cc:276] Timestamp: P: 1781639914656244 usec, L: 0
0616 19:58:34.658011 (+ 24us) op_driver.cc:348] REPLICATION: starting
0616 19:58:34.658798 (+ 787us) log.cc:844] Serialized 6353 byte log entry
0616 19:58:35.006950 (+348152us) op_driver.cc:464] REPLICATION: finished
0616 19:58:35.101747 (+ 94797us) write_op.cc:301] APPLY: starting
0616 19:58:35.104908 (+ 3161us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:35.124329 (+ 19421us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:35.124471 (+ 142us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:35.127612 (+ 3141us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:35.127885 (+ 273us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0616 19:58:35.127935 (+ 50us) write_op.cc:312] APPLY: finished
0616 19:58:35.128361 (+ 426us) log.cc:844] Serialized 112 byte log entry
0616 19:58:35.138345 (+ 9984us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:35.147147 (+ 8802us) write_op.cc:454] Released schema lock
0616 19:58:35.170924 (+ 23777us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:35.190404 (+ 19480us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:35.191744 (+ 1340us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":53233,"apply.run_cpu_time_us":4580,"apply.run_wall_time_us":91308,"num_ops":16,"prepare.queue_time_us":18531,"prepare.run_cpu_time_us":4854,"prepare.run_wall_time_us":6562,"replication_time_us":348771,"thread_start_us":66171,"threads_started":2}]]}
I20260616 19:58:35.483774 9597 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:35.525269 9676 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.525379 8009 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:35.525887 9676 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.532908 9732 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:35.533675 9732 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:35.537326 9715 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.538028 9715 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.500586 9377 consensus_queue.cc:1059] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260616 19:58:35.544961 9377 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:58:35.545321 9377 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:35.546185 9377 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.547379 9377 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:35.548547 9462 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:35.551232 9531 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.551577 9740 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.490352 9694 consensus_queue.cc:1059] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260616 19:58:35.552726 9694 consensus_queue.cc:1243] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260616 19:58:35.542027 9549 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:35.555084 9549 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.556387 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.557070 9549 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.557650 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:35.464442 9384 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.560632 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.562345 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.569423 7966 raft_consensus.cc:2410] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:58:35.558534 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:35.561790 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 59909.8us
I20260616 19:58:35.570125 7935 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.521004 9805 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:35.613982 7935 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:35.534559 9732 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.618160 9597 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:35.618698 9597 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:35.618865 9583 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:58:35.619313 9583 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:35.619699 9583 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:35.619561 9597 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.572888 9676 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.631193 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 29) took 1503 ms. Trace:
W20260616 19:58:35.624666 9788 log.cc:927] Time spent T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260616 19:58:35.615522 9690 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:35.631652 7864 rpcz_store.cc:276] 0616 19:58:34.127727 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:34.132306 (+ 4579us) service_pool.cc:224] Handling call
0616 19:58:35.631130 (+1498824us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":225,"mutex_wait_us":75999,"reported_tablets":2}
I20260616 19:58:35.632972 9690 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.616643 7935 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.615253 9689 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:35.622857 9715 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:35.635470 7899 debug-util.cc:398] Leaking SignalData structure 0x7b080003cd60 after lost signal to thread 6785
W20260616 19:58:35.641039 7899 debug-util.cc:398] Leaking SignalData structure 0x7b08001f21a0 after lost signal to thread 7902
I20260616 19:58:35.641043 9690 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:35.642308 7899 debug-util.cc:398] Leaking SignalData structure 0x7b0800272100 after lost signal to thread 7995
W20260616 19:58:35.643477 7899 debug-util.cc:398] Leaking SignalData structure 0x7b0800354400 after lost signal to thread 8069
W20260616 19:58:35.643783 7899 debug-util.cc:398] Leaking SignalData structure 0x7b0800394b60 after lost signal to thread 8070
I20260616 19:58:35.647194 9583 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.651005 9597 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:58:35.665201 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 48014.7us
I20260616 19:58:35.615896 9814 consensus_queue.cc:1059] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.088s
I20260616 19:58:35.638056 9695 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:35.669543 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.661792 9741 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.655484 9815 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:35.663704 9798 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.677549 9720 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.674962 9814 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:35.673121 9741 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.682325 8116 raft_consensus.cc:1217] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Deduplicated request from leader. Original: 5.3->[6.4-6.4] Dedup: 6.4->[]
I20260616 19:58:35.681918 9804 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.685693 8116 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260616 19:58:35.686384 9804 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.687456 9583 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.688973 9804 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.692494 9741 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.683187 9720 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.694717 9720 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.672019 9695 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:35.680253 7899 debug-util.cc:398] Leaking SignalData structure 0x7b080036cbc0 after lost signal to thread 8143
I20260616 19:58:35.694605 7935 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.702607 9384 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.702880 9815 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:35.708360 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.708699 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.707840 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.709782 9814 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:35.676970 9798 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.703761 9695 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.713734 9798 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:35.716833 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:35.719583 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:35.726559 9816 log.cc:927] Time spent T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.102s user 0.001s sys 0.000s
I20260616 19:58:35.728679 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.729028 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:35.729821 7970 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260616 19:58:35.732985 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.734247 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:35.734602 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.734789 7968 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:35.733683 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:35.736073 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:35.735948 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:35.738739 9822 ts_tablet_manager.cc:933] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:58:35.740051 7969 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:35.739950 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.745898 9635 raft_consensus.cc:3055] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:58:35.724411 9814 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:35.746434 9635 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:35.746534 9814 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.748754 9814 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.747596 9635 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.750645 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.754609 7968 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:35.758500 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.761129 9635 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:58:35.762844 7899 debug-util.cc:398] Leaking SignalData structure 0x7b08001b8a80 after lost signal to thread 9654
W20260616 19:58:35.763283 7899 debug-util.cc:398] Leaking SignalData structure 0x7b08002227a0 after lost signal to thread 9681
W20260616 19:58:35.763865 7899 debug-util.cc:398] Leaking SignalData structure 0x7b08002dfe80 after lost signal to thread 9729
I20260616 19:58:35.764379 8112 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.765489 8112 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:35.769878 8011 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.774051 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.775772 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:35.777558 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.776546 7969 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:35.780128 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.788374 8116 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:58:35.808777 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.812668 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:35.812512 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.814687 8112 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:35.817564 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.821820 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.772574 9597 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:58:35.823897 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.833652 8009 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:35.840858 9377 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:35.840449 9462 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:35.841555 9377 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:35.842769 9377 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.846743 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.847350 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:35.852981 7934 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.854465 7934 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.856722 9384 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:35.857582 9384 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:58:35.859068 7934 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.834968 9759 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.864656 9759 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.826253 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.866214 7934 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:35.866482 8011 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:35.867833 9694 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:35.868425 9597 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:35.867460 9814 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:35.870899 9814 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:35.829433 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:35.881938 8043 raft_consensus.cc:2410] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:58:35.866669 9759 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:35.887029 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.903530 9384 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:35.923632 9804 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.924258 9804 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.926692 9804 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.930276 8043 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260616 19:58:35.930819 8043 pending_rounds.cc:85] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 4
I20260616 19:58:35.931077 8043 pending_rounds.cc:107] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260616 19:58:35.931308 8043 raft_consensus.cc:2889] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:58:35.935086 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:35.935926 7968 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:35.937745 8011 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:35.938791 9804 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:58:35.939232 9804 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:35.939662 9804 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:35.952435 9812 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:35.991036 9804 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:35.994656 9804 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:35.995579 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:35.999328 9805 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:36.030632 7966 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:36.030856 8084 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.032564 8038 raft_consensus.cc:1240] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:36.032022 8011 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:36.044530 9377 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:36.051378 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.045217 9294 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:36.060335 7966 raft_consensus.cc:1240] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:36.062968 8084 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.072202 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.072216 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:36.073886 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:36.077098 8116 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:36.084537 9395 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:36.085006 8084 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.086553 8084 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.081737 8116 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.063249 9828 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:36.063515 9377 raft_consensus.cc:697] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:36.079128 9531 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:58:36.106298 7966 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:36.097890 9715 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:58:36.119704 9720 consensus_queue.cc:1059] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:36.134183 9676 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:36.103106 9822 tablet_copy_client.cc:323] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
I20260616 19:58:36.137874 9549 raft_consensus.cc:3055] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:36.157299 9377 consensus_queue.cc:237] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:36.159327 8819 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:36.161582 9742 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:36.156926 9549 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:36.166198 9549 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:36.167485 9549 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:36.167910 8040 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.171185 8084 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.175483 9462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:41278 (request call id 0) took 1032 ms. Trace:
I20260616 19:58:36.175930 9462 rpcz_store.cc:276] 0616 19:58:35.147375 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:35.147551 (+ 176us) service_pool.cc:224] Handling call
0616 19:58:35.196295 (+ 48744us) log.cc:844] Serialized 506 byte log entry
0616 19:58:35.310229 (+113934us) spinlock_profiling.cc:243] Waited 964 us on lock 0x7b54000f58a0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f25e522 00007fe86f23f28a 00007fe86f246133 00007fe86f2460b9 00007fe86f246049 00007fe86f246011 00007fe86f2451dd 0000000000389354 0000000000389199 00007fe86c233ab5
0616 19:58:35.354961 (+ 44732us) spinlock_profiling.cc:243] Waited 989 us on lock 0x7b18003e1be0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:35.421747 (+ 66786us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b18003e1be0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:36.175361 (+753614us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":204,"mutex_wait_us":5487,"raft.queue_time_us":5925,"raft.run_cpu_time_us":3088,"raft.run_wall_time_us":82181,"spinlock_wait_cycles":12624384,"thread_start_us":35584,"threads_started":1,"wal-append.queue_time_us":62637}
W20260616 19:58:36.189788 7916 auto_rebalancer.cc:591] Failed to schedule move for tablet 69c80791d0c44cb0b616bf402ae91b46: Illegal state: Replica b022cac8cb5e46b7a39a748e81e257a7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
W20260616 19:58:36.194406 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:36.195551 9740 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:36.208734 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:36.217185 7980 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet 0d19ee585ead4942a40096f7aef3ba50 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
I20260616 19:58:36.217831 7980 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet 0d19ee585ead4942a40096f7aef3ba50 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-0d19ee585ead4942a40096f7aef3ba50
I20260616 19:58:36.223701 8042 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:36.226920 7980 tablet_copy_source_session.cc:215] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:36.241109 7963 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:36.242553 9805 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:36.248025 8040 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:36.249569 9786 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:36.242978 7864 catalog_manager.cc:5696] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 5 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:36.241442 9822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d19ee585ead4942a40096f7aef3ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:36.254482 7961 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:36.258625 9377 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:36.278486 9822 tablet_copy_client.cc:806] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:36.279310 9822 tablet_copy_client.cc:670] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:36.279798 7961 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:36.312031 8116 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:36.314049 9649 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:36.328751 7864 catalog_manager.cc:5696] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
W20260616 19:58:36.330641 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:36.321849 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 66893.3us
I20260616 19:58:36.332919 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.339687 8110 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:36.333389 7864 catalog_manager.cc:5696] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:36.353127 9822 tablet_copy_client.cc:538] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:36.358098 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 131) took 4327 ms. Trace:
I20260616 19:58:36.358536 7865 rpcz_store.cc:276] 0616 19:58:32.030842 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:32.031026 (+ 184us) service_pool.cc:224] Handling call
0616 19:58:36.358035 (+4327009us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49214,"reported_tablets":41}
I20260616 19:58:36.374082 8110 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:36.381841 9531 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:36.384266 9377 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:36.382931 7864 catalog_manager.cc:5696] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:36.392128 9732 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:36.393654 7864 catalog_manager.cc:5696] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:36.377892 9740 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:36.395434 7864 catalog_manager.cc:5696] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:36.397502 7864 catalog_manager.cc:5696] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:36.398731 7864 catalog_manager.cc:5696] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 2 to 3, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:36.404851 9786 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:36.413995 9822 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:36.424315 8037 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:36.468824 8038 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260616 19:58:36.488715 8082 consensus_peers.cc:597] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43378ff312f94246811895976a8715d6. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:36.489490 9790 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:36.509171 7963 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:36.531319 7961 raft_consensus.cc:2955] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:36.532199 9294 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260616 19:58:36.561820 9294 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.565665 7852 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet cf8a24d3c225486fa9270926ffe6a40e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:36.567541 8037 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.575927 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:36.693885 8119 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:36.695734 9822 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:36.728255 9822 tablet_bootstrap.cc:492] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:36.729059 9822 ts_tablet_manager.cc:1403] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.315s user 0.022s sys 0.018s
I20260616 19:58:36.697892 7963 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.703543 7868 catalog_manager.cc:5696] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 8, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:36.752825 9822 raft_consensus.cc:359] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:36.753835 9822 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:36.754550 9822 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:36.756718 7864 catalog_manager.cc:5696] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:36.758826 7864 catalog_manager.cc:5696] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.760315 7864 catalog_manager.cc:5696] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 5 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:36.762045 7864 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 2 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:36.763278 7864 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 4, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:36.764799 8113 raft_consensus.cc:1275] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:36.765833 7866 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New cstate: current_term: 6 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:36.769420 9637 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:36.780789 8101 tablet_service.cc:1558] Processing DeleteTablet for tablet cf8a24d3c225486fa9270926ffe6a40e with delete_type TABLET_DATA_TOMBSTONED (TS 6827184e3cef42c2ada15b2d31afee4b not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:36.783319 9822 ts_tablet_manager.cc:1434] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.054s user 0.007s sys 0.000s
I20260616 19:58:36.791585 7980 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-0d19ee585ead4942a40096f7aef3ba50 received from {username='slave'} at 127.0.0.1:37398
I20260616 19:58:36.792098 7980 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-0d19ee585ead4942a40096f7aef3ba50 on tablet 0d19ee585ead4942a40096f7aef3ba50 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:36.794245 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:36.794898 7963 raft_consensus.cc:606] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.808239 7966 raft_consensus.cc:1275] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260616 19:58:36.808310 8082 consensus_peers.cc:597] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba7d833edb374b10abaeb128d2084328. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:36.809420 7961 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260616 19:58:36.811352 9855 tablet_replica.cc:333] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: stopping tablet replica
I20260616 19:58:36.811041 9790 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:36.812498 9855 raft_consensus.cc:2243] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:36.813249 9855 raft_consensus.cc:2272] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:36.817018 9855 ts_tablet_manager.cc:1916] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:36.823401 9377 consensus_queue.cc:1048] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:36.836423 9855 ts_tablet_manager.cc:1929] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.6
I20260616 19:58:36.837074 9855 log.cc:1199] T cf8a24d3c225486fa9270926ffe6a40e P 6827184e3cef42c2ada15b2d31afee4b: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root/wals/cf8a24d3c225486fa9270926ffe6a40e
I20260616 19:58:36.840144 8119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:36.839408 7853 catalog_manager.cc:5027] TS 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): tablet cf8a24d3c225486fa9270926ffe6a40e (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:36.842144 8040 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260616 19:58:36.840837 8119 raft_consensus.cc:606] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.853070 9790 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:36.853951 8119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:36.854624 8119 raft_consensus.cc:606] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.855201 8119 raft_consensus.cc:614] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:58:36.857817 8119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:36.862525 8119 raft_consensus.cc:606] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.870204 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:36.870836 8116 raft_consensus.cc:606] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.873605 9740 raft_consensus.cc:993] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:36.874100 9740 raft_consensus.cc:1081] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:36.878556 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:36.879179 8116 raft_consensus.cc:606] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:36.880407 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260616 19:58:36.882216 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:36.883057 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:36.894124 8038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:36.894848 8038 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:36.895265 8038 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:36.913897 9402 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.943140 8038 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:36.946095 8037 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:36.961037 9822 ts_tablet_manager.cc:933] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:36.947604 7865 catalog_manager.cc:5696] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 4 to 5, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:36.988796 9294 raft_consensus.cc:993] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:36.989425 9790 raft_consensus.cc:1081] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:36.996670 8044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:36.997304 8044 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:36.997668 8044 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:37.009778 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:37.010485 7961 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:37.014531 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.015161 8119 raft_consensus.cc:3055] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:37.015470 8119 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:37.019135 8044 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:37.016389 8119 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.053146 8119 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:37.056216 8038 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.057881 7961 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:37.072608 8044 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:37.074028 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:58:37.076548 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.077195 8116 raft_consensus.cc:3055] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:37.077533 8116 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:37.078253 8116 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:37.079264 8116 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:37.080998 8011 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:37.075189 8819 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:37.088768 8119 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:37.092516 9377 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:37.093152 9377 raft_consensus.cc:697] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:37.094135 9377 consensus_queue.cc:237] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.094264 8819 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:37.096606 9822 tablet_copy_client.cc:323] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.131:40887
I20260616 19:58:37.099169 8008 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:37.100026 9805 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:37.100627 9805 raft_consensus.cc:697] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:37.110776 8116 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:37.110208 7966 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:37.116765 9384 raft_consensus.cc:993] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:37.117336 9814 raft_consensus.cc:1081] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:37.137465 8129 tablet_copy_service.cc:140] P 6827184e3cef42c2ada15b2d31afee4b: Received BeginTabletCopySession request for tablet 43378ff312f94246811895976a8715d6 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:34110)
I20260616 19:58:37.138063 8129 tablet_copy_service.cc:161] P 6827184e3cef42c2ada15b2d31afee4b: Beginning new tablet copy session on tablet 43378ff312f94246811895976a8715d6 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:34110: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-43378ff312f94246811895976a8715d6
I20260616 19:58:37.145399 8044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:58:37.146155 8044 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:37.146586 8044 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:37.170279 8044 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.113919 9805 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:58:37.213569 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 58737.8us
I20260616 19:58:37.182047 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:37.195505 8129 tablet_copy_source_session.cc:215] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:37.196368 8044 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.235601 9637 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:37.236254 9637 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.220517 7966 raft_consensus.cc:3055] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:37.249140 7966 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:37.262257 7966 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.267335 7966 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:37.286336 7966 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
W20260616 19:58:37.327584 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 167254us
I20260616 19:58:37.280134 9384 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:58:37.378186 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 87177.2us
W20260616 19:58:37.372995 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 133886us
I20260616 19:58:37.389317 8011 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:37.354512 9384 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.396513 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:37.404610 9872 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:37.405212 9872 raft_consensus.cc:697] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:37.406330 9872 consensus_queue.cc:237] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.409777 9384 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.410392 7966 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:58:37.413245 9822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43378ff312f94246811895976a8715d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:37.444653 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:37.445386 8044 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:58:37.445396 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 112904us
I20260616 19:58:37.483094 7935 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:37.441370 9637 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.519325 9586 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:58:37.529960 9586 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:37.530337 9586 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:37.533221 9864 raft_consensus.cc:993] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:37.540522 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:37.541167 8113 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:58:37.535475 9864 raft_consensus.cc:1081] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:37.555415 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 132) took 1025 ms. Trace:
I20260616 19:58:37.557595 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.558207 8112 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:37.558525 8819 raft_consensus.cc:1217] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4] Dedup: 4.4->[]
I20260616 19:58:37.557859 7868 catalog_manager.cc:5696] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:37.565400 8112 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:58:37.555878 7865 rpcz_store.cc:276] 0616 19:58:36.530433 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:36.532465 (+ 2032us) service_pool.cc:224] Handling call
0616 19:58:37.555353 (+1022888us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:37.090707 (+ 0us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b54000acc28. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8756a2c21 00007fe8756a188c 00007fe872d3195c 00007fe872d1cf26 00007fe872d1cbd8 00007fe872d21871 00007fe872d21829 00007fe872d217b9 00007fe872d21781 00007fe872d209ad 0000000000389354
0616 19:58:37.090840 (+ 133us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:37.104404 (+ 13564us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:37.105528 (+ 1124us) write_op.cc:435] Acquired schema lock
0616 19:58:37.105546 (+ 18us) tablet.cc:662] Decoding operations
0616 19:58:37.132876 (+ 27330us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:37.132944 (+ 68us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:37.132992 (+ 48us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:37.133209 (+ 217us) tablet.cc:701] Row locks acquired
0616 19:58:37.133230 (+ 21us) write_op.cc:260] PREPARE: finished
0616 19:58:37.133374 (+ 144us) write_op.cc:270] Start()
0616 19:58:37.133583 (+ 209us) write_op.cc:276] Timestamp: P: 1781639917133327 usec, L: 0
0616 19:58:37.133613 (+ 30us) op_driver.cc:348] REPLICATION: starting
0616 19:58:37.134309 (+ 696us) log.cc:844] Serialized 540 byte log entry
0616 19:58:37.138412 (+ 4103us) op_driver.cc:464] REPLICATION: finished
0616 19:58:37.138859 (+ 447us) write_op.cc:301] APPLY: starting
0616 19:58:37.138942 (+ 83us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:37.139063 (+ 121us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:37.139088 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:37.139353 (+ 265us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:37.139520 (+ 167us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:37.139571 (+ 51us) write_op.cc:312] APPLY: finished
0616 19:58:37.139776 (+ 205us) log.cc:844] Serialized 22 byte log entry
0616 19:58:37.139969 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:37.140073 (+ 104us) write_op.cc:454] Released schema lock
0616 19:58:37.140117 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:37.140505 (+ 388us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:37.140540 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":181706,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":250,"apply.run_cpu_time_us":1586,"apply.run_wall_time_us":1586,"num_ops":1,"prepare.queue_time_us":3440,"prepare.run_cpu_time_us":6396,"prepare.run_wall_time_us":70208,"replication_time_us":4647,"spinlock_wait_cycles":30490112}]]}
I20260616 19:58:37.595484 9879 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:37.600035 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:37.609084 9879 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.611162 9879 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.609563 8119 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:58:37.595860 9586 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.631251 9586 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:37.645118 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 149785us
I20260616 19:58:37.647826 8011 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:37.648701 9885 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:58:37.649109 9885 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:37.649492 9885 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:37.650655 9822 tablet_copy_client.cc:806] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:37.651254 9822 tablet_copy_client.cc:670] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:37.668453 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:37.669077 8119 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:37.670070 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.670841 8112 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:37.677333 9885 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:37.679314 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 179552us
I20260616 19:58:37.682873 9893 raft_consensus.cc:993] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:37.684444 9893 raft_consensus.cc:1081] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:37.690160 8112 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:58:37.679430 9885 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.692684 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.704382 9822 tablet_copy_client.cc:538] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:37.710907 8115 raft_consensus.cc:2368] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:58:37.712786 7865 catalog_manager.cc:5696] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 3 to 4, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:37.714646 7865 catalog_manager.cc:5696] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:37.708806 7934 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:37.720137 7966 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:37.720639 8009 leader_election.cc:400] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:58:37.720983 8009 leader_election.cc:403] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260616 19:58:37.721560 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:37.721794 9894 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:37.722287 7963 raft_consensus.cc:2368] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:58:37.730003 9897 ts_tablet_manager.cc:933] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.732065 7865 catalog_manager.cc:5696] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 6 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 7 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:37.734923 9879 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:58:37.740437 9879 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:58:37.740828 9879 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:37.743647 9885 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:37.745926 8011 leader_election.cc:400] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:58:37.748359 9879 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.755053 9879 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.756395 9894 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:58:37.753114 9822 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:37.757855 8115 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:37.765722 9897 tablet_copy_client.cc:323] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.131:40887
I20260616 19:58:37.770183 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:37.771186 8129 tablet_copy_service.cc:140] P 6827184e3cef42c2ada15b2d31afee4b: Received BeginTabletCopySession request for tablet ba7d833edb374b10abaeb128d2084328 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:34110)
I20260616 19:58:37.771179 9885 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:37.771762 8129 tablet_copy_service.cc:161] P 6827184e3cef42c2ada15b2d31afee4b: Beginning new tablet copy session on tablet ba7d833edb374b10abaeb128d2084328 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:34110: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-ba7d833edb374b10abaeb128d2084328
I20260616 19:58:37.781563 8129 tablet_copy_source_session.cc:215] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:37.790977 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:37.791793 8044 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:37.795611 9897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba7d833edb374b10abaeb128d2084328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:37.797896 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:37.805728 8044 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:58:37.810120 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:37.810755 8041 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 7
I20260616 19:58:37.813632 9867 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:58:37.814179 9867 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:37.814960 9867 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.818567 8041 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:58:37.822257 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 52160.1us
I20260616 19:58:37.844961 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.845582 8112 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:37.853521 8041 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:37.854097 8041 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:37.854439 8041 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:37.861896 8112 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:37.865082 8038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:37.865680 8038 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:37.866053 8038 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:37.866251 7934 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:37.867656 9890 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:37.868103 9890 raft_consensus.cc:697] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:37.868921 9890 consensus_queue.cc:237] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.872560 8038 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.879473 8038 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.877400 9822 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:37.897125 9822 tablet_bootstrap.cc:492] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:37.897838 9822 ts_tablet_manager.cc:1403] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.168s user 0.020s sys 0.008s
I20260616 19:58:37.877111 8041 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.902694 7963 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:37.905321 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.905620 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:37.905951 8115 raft_consensus.cc:3055] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:37.906394 7961 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:37.906798 8115 raft_consensus.cc:740] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:37.911063 8041 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:37.911226 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:37.911787 8112 raft_consensus.cc:3055] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:37.912099 8112 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:37.912829 8112 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.913870 8112 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:37.918690 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:37.919539 7963 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:37.925315 8112 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:37.927002 7961 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:37.928340 8011 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:37.927137 8009 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:37.909430 8115 consensus_queue.cc:260] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.930529 9894 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260616 19:58:37.944842 9897 tablet_copy_client.cc:806] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:37.945508 9897 tablet_copy_client.cc:670] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:37.952458 8115 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:37.954049 9869 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:37.954591 9869 raft_consensus.cc:697] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:37.955451 9869 consensus_queue.cc:237] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.956341 9880 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:37.956820 9880 raft_consensus.cc:697] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:37.961045 8115 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:37.961369 7963 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:37.957784 9880 consensus_queue.cc:237] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:37.952521 9897 tablet_copy_client.cc:538] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:37.975541 7868 catalog_manager.cc:5696] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 7, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:37.979092 8819 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:58:37.990927 7868 catalog_manager.cc:5696] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:37.987573 9822 raft_consensus.cc:359] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:37.997756 9822 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:37.998914 9822 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:38.006028 9822 ts_tablet_manager.cc:1434] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.108s user 0.009s sys 0.000s
I20260616 19:58:38.017457 9586 raft_consensus.cc:1064] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:38.021207 8129 tablet_copy_service.cc:342] P 6827184e3cef42c2ada15b2d31afee4b: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-43378ff312f94246811895976a8715d6 received from {username='slave'} at 127.0.0.1:34110
I20260616 19:58:38.021754 8129 tablet_copy_service.cc:434] P 6827184e3cef42c2ada15b2d31afee4b: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-43378ff312f94246811895976a8715d6 on tablet 43378ff312f94246811895976a8715d6 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:38.036562 8115 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:58:38.024786 9586 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:38.071976 8041 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:38.064195 9637 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260616 19:58:38.080986 9897 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:38.089713 8038 raft_consensus.cc:1240] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:38.088196 9637 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260616 19:58:38.143213 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:38.142400 7868 catalog_manager.cc:5696] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:38.144454 7868 catalog_manager.cc:5696] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260616 19:58:38.226023 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 141953us
I20260616 19:58:38.248893 8112 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:38.262578 9880 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260616 19:58:38.292723 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 53430.8us
I20260616 19:58:38.303987 8817 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260616 19:58:38.305594 9915 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:38.310726 8111 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260616 19:58:38.329996 7961 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:38.330739 7961 raft_consensus.cc:606] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:38.327723 9867 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260616 19:58:38.340574 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:38.341173 8115 raft_consensus.cc:606] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:38.360743 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:38.361356 8116 raft_consensus.cc:606] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:38.371680 9870 raft_consensus.cc:993] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:38.379787 9866 raft_consensus.cc:1081] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:38.382694 8037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:38.383441 8037 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:38.383850 8037 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
W20260616 19:58:38.401278 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 43902.7us
I20260616 19:58:38.420183 8037 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:38.420811 8041 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260616 19:58:38.443115 8037 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:38.445129 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f03211770a74604a786608de5062cc3"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:38.445760 7963 raft_consensus.cc:606] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:38.444725 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 83594.4us
I20260616 19:58:38.479913 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:38.480563 8112 raft_consensus.cc:606] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:38.480940 8112 raft_consensus.cc:614] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:58:38.487452 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:38.489231 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:38.490046 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260616 19:58:38.523437 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 151548us
I20260616 19:58:38.550071 8115 raft_consensus.cc:1275] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:38.492192 9883 raft_consensus.cc:993] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:38.561144 8119 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:38.561250 8116 raft_consensus.cc:1275] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:38.564723 7967 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:38.558509 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:38.566129 8112 raft_consensus.cc:3055] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:38.566447 8112 raft_consensus.cc:740] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:38.572494 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:38.567169 8112 consensus_queue.cc:260] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:38.573354 7963 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:38.573800 8112 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:38.491007 8820 raft_consensus.cc:1217] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4] Dedup: 4.4->[]
I20260616 19:58:38.587103 8112 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:38.615696 7963 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:58:38.582470 9883 raft_consensus.cc:1081] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
W20260616 19:58:38.640674 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 61050.1us
I20260616 19:58:38.589376 9894 consensus_queue.cc:1048] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260616 19:58:38.646940 8037 raft_consensus.cc:1275] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:38.647502 8009 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:38.656441 8037 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260616 19:58:38.586787 9910 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260616 19:58:38.589133 9875 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260616 19:58:38.668679 9886 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:38.699298 9886 raft_consensus.cc:697] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:38.699994 9920 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:38.700237 9886 consensus_queue.cc:237] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:38.704627 9880 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:38.677309 9890 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260616 19:58:38.755020 9897 log.cc:927] Time spent T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.086s user 0.001s sys 0.000s
W20260616 19:58:38.765573 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 183605us
I20260616 19:58:38.769575 9919 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260616 19:58:38.752324 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 81990.5us
I20260616 19:58:38.825644 7967 raft_consensus.cc:1275] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:38.835024 8040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:38.835597 8040 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:38.835940 8040 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:38.850260 8040 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:38.852604 8040 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:38.861779 9867 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:38.865182 8116 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260616 19:58:38.888445 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:38.889065 8119 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:38.889389 8119 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:38.908488 8119 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:38.909951 8119 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:38.919700 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:38.920588 7969 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:38.984248 8119 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:38.985785 7969 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:38.915148 9869 consensus_queue.cc:1048] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260616 19:58:38.991822 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:39.036851 9936 log.cc:927] Time spent T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20260616 19:58:39.062551 9897 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:39.063853 9897 tablet_bootstrap.cc:492] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:39.064793 9897 ts_tablet_manager.cc:1403] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.984s user 0.036s sys 0.004s
W20260616 19:58:39.074357 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 182793us
W20260616 19:58:39.075780 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 84788.9us
W20260616 19:58:39.075537 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 74716.5us
I20260616 19:58:39.033497 9886 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
W20260616 19:58:39.174783 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 92669.5us
W20260616 19:58:39.188930 9940 log.cc:927] Time spent T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
W20260616 19:58:39.198592 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 120897us
I20260616 19:58:39.167760 9920 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.066s
I20260616 19:58:39.230141 9886 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:39.232733 9886 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:39.225703 9930 raft_consensus.cc:993] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
W20260616 19:58:39.274919 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 95635.1us
W20260616 19:58:39.276443 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 71112.7us
W20260616 19:58:39.290940 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 59841.4us
I20260616 19:58:39.275767 9918 raft_consensus.cc:1081] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
W20260616 19:58:39.307168 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 220901us
W20260616 19:58:39.347796 9950 log.cc:927] Time spent T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.056s user 0.000s sys 0.001s
W20260616 19:58:39.350873 9944 log.cc:927] Time spent T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.173s user 0.001s sys 0.002s
I20260616 19:58:39.357573 7970 raft_consensus.cc:1275] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260616 19:58:39.377560 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 48749us
I20260616 19:58:39.333179 9897 raft_consensus.cc:359] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:39.402729 8111 raft_consensus.cc:1275] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:39.416646 8111 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:39.404536 9897 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:39.425158 8116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:39.426679 8116 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:39.426982 8116 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:39.438297 8116 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:39.451084 8113 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:39.503209 9897 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:39.506860 9897 ts_tablet_manager.cc:1434] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.442s user 0.009s sys 0.003s
W20260616 19:58:39.535950 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 133896us
I20260616 19:58:39.625594 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:39.658017 8116 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:39.663697 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 221591us
W20260616 19:58:39.664418 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 222100us
I20260616 19:58:39.671339 9889 raft_consensus.cc:993] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:39.676606 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:39.677242 7967 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:39.677546 7967 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:39.678256 7967 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:39.678927 9966 raft_consensus.cc:1064] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:39.679131 7967 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:39.689281 9921 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:39.690073 9921 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:39.688858 7967 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:39.696635 8085 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:39.697400 9878 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:39.697919 9878 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:39.698765 9878 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:39.704799 9966 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:39.717075 8820 raft_consensus.cc:1217] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260616 19:58:39.721387 9962 consensus_queue.cc:1048] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:39.749176 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:39.751080 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 91553.4us
W20260616 19:58:39.804312 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 51034.9us
W20260616 19:58:39.772572 9971 log.cc:927] Time spent T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.133s user 0.001s sys 0.000s
I20260616 19:58:39.748232 9921 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:39.814649 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 61447.5us
I20260616 19:58:39.822903 8129 tablet_copy_service.cc:342] P 6827184e3cef42c2ada15b2d31afee4b: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-ba7d833edb374b10abaeb128d2084328 received from {username='slave'} at 127.0.0.1:34110
I20260616 19:58:39.824331 9965 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:58:39.828437 9977 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.076s user 0.000s sys 0.001s
I20260616 19:58:39.835400 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:39.806519 9953 raft_consensus.cc:1081] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
W20260616 19:58:39.854761 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:39.862107 8785 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:39.863066 9896 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:39.807747 9805 consensus_queue.cc:1048] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260616 19:58:39.852583 8129 tablet_copy_service.cc:434] P 6827184e3cef42c2ada15b2d31afee4b: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-ba7d833edb374b10abaeb128d2084328 on tablet ba7d833edb374b10abaeb128d2084328 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:39.860320 9965 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:39.870901 9965 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:39.857647 9883 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:39.875752 7970 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260616 19:58:39.877398 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:58:39.878505 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45845.9us
I20260616 19:58:39.872824 9883 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:39.885210 8820 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:39.887050 9883 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:39.926375 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49198.6us
I20260616 19:58:39.882879 7866 catalog_manager.cc:5696] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 2 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:40.015785 8817 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260616 19:58:40.022464 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 51338.4us
I20260616 19:58:39.962409 9878 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260616 19:58:40.032320 9933 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:40.032888 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:58:40.033483 9978 log.cc:927] Time spent T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.132s user 0.001s sys 0.000s
W20260616 19:58:40.013175 9941 log.cc:927] Time spent T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.103s user 0.000s sys 0.000s
I20260616 19:58:40.034818 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:40.037329 7967 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:40.070020 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:40.047307 9880 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:40.082712 8041 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:40.091572 8038 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260616 19:58:40.068679 9948 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:40.110805 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:40.115607 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44475.2us
I20260616 19:58:40.122678 8041 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:40.124258 8038 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: []
I20260616 19:58:40.095342 9880 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.145324 7970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:40.151917 7970 raft_consensus.cc:606] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:40.163903 8037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f03211770a74604a786608de5062cc3"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:58:40.164644 8037 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:40.165019 8037 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:40.165517 9880 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:58:40.213402 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:40.108054 9948 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:40.280903 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 100224us
I20260616 19:58:40.283944 8037 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:40.289613 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 121473us
I20260616 19:58:40.290016 8037 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:40.296761 9948 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:40.299350 9963 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:40.299913 9963 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.302022 9963 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:40.314687 9917 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:40.317332 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:40.317539 7970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:40.323966 7970 raft_consensus.cc:606] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:40.325446 8037 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:40.327499 9925 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260616 19:58:40.325536 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:40.332374 7970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:40.332959 7970 raft_consensus.cc:606] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:40.337355 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:40.356514 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:40.357322 8112 raft_consensus.cc:606] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:40.348826 9885 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:40.377913 9885 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:40.383471 9885 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:40.384954 9956 raft_consensus.cc:1064] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:40.390887 9669 raft_consensus.cc:993] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:40.391390 9669 raft_consensus.cc:1081] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:40.387490 9956 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:40.402390 8112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:40.403023 8112 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:40.403497 8112 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:40.412516 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:40.416839 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:40.417423 8110 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:40.418897 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:40.426512 7967 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:40.430238 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:40.440873 8110 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:40.430619 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:40.447397 8112 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.454867 8112 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:40.449574 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 77147.7us
I20260616 19:58:40.456945 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:40.457767 9886 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260616 19:58:40.459537 9987 log.cc:927] Time spent T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.178s user 0.000s sys 0.001s
I20260616 19:58:40.460065 9885 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:40.462991 9586 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:40.463527 9586 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.466020 9586 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:40.467113 8110 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:58:40.470660 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:40.472102 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:40.471052 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:40.473063 7961 raft_consensus.cc:3055] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:40.473379 7961 raft_consensus.cc:740] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:40.474107 7961 consensus_queue.cc:260] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.473477 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:40.478106 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:40.490365 8085 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:40.495088 9865 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:58:40.490370 9892 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:40.499249 9892 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.500625 8085 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:40.501497 9931 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:58:40.501616 9892 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:40.503307 9933 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:40.513396 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:40.513936 7963 raft_consensus.cc:3055] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:40.514211 7963 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:40.514871 7963 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.517277 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:40.518031 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:40.520443 7963 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:40.530666 7963 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:40.536011 8085 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:40.535972 8009 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:58:40.547384 9994 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.163s user 0.001s sys 0.000s
I20260616 19:58:40.558125 8009 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:40.561645 7970 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260616 19:58:40.565642 8117 raft_consensus.cc:1240] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:40.568301 8817 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260616 19:58:40.568655 9867 raft_consensus.cc:993] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:40.569144 9867 raft_consensus.cc:1081] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:40.569916 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:40.570946 8112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:40.571477 8112 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:40.571843 8112 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:40.574318 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:40.575423 9962 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:40.577477 8011 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:40.579135 9892 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:58:40.561568 9880 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:58:40.583837 9916 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:40.561785 9954 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:40.584749 9954 raft_consensus.cc:697] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:40.585714 9954 consensus_queue.cc:237] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.587368 8112 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.577890 9933 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260616 19:58:40.592083 7961 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:40.592624 9916 raft_consensus.cc:697] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:40.599469 9916 consensus_queue.cc:237] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.559846 9872 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260616 19:58:40.608376 9963 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:40.608381 8011 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:40.618467 7961 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:40.609881 9951 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:58:40.623484 9956 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.642153 8112 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:40.635294 7969 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.656982 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:40.685642 7961 raft_consensus.cc:3055] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:40.690551 8037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:40.691243 8037 raft_consensus.cc:606] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:40.686123 7961 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:40.695439 7961 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.699380 7961 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:40.701166 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:40.719175 7961 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:40.721841 8085 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:40.731906 9916 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:40.734769 9916 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:40.735715 9916 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.765980 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 40) took 1082 ms. Trace:
I20260616 19:58:40.766605 7864 rpcz_store.cc:276] 0616 19:58:39.683577 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:39.695137 (+ 11560us) service_pool.cc:224] Handling call
0616 19:58:40.151635 (+456498us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b5400002490. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6c02 00007fe87513a659 00007fe8750d28b6 00007fe8750a3968 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc
0616 19:58:40.550908 (+399273us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:40.607564 (+ 56656us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b5400002490. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f7190 00007fe875098456 00007fe8751a296e 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:58:40.765890 (+158326us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":510408064}
I20260616 19:58:40.777048 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 138) took 1864 ms. Trace:
I20260616 19:58:40.778649 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:40.782104 7868 catalog_manager.cc:5696] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.784782 7868 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:40.794124 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:40.777650 7866 rpcz_store.cc:276] 0616 19:58:38.912605 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:38.912778 (+ 173us) service_pool.cc:224] Handling call
0616 19:58:40.776965 (+1864187us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:40.124012 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:40.124845 (+ 833us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:40.124901 (+ 56us) write_op.cc:435] Acquired schema lock
0616 19:58:40.124929 (+ 28us) tablet.cc:662] Decoding operations
0616 19:58:40.125352 (+ 423us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:40.125431 (+ 79us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:40.125463 (+ 32us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:40.125662 (+ 199us) tablet.cc:701] Row locks acquired
0616 19:58:40.125682 (+ 20us) write_op.cc:260] PREPARE: finished
0616 19:58:40.125788 (+ 106us) write_op.cc:270] Start()
0616 19:58:40.145031 (+ 19243us) write_op.cc:276] Timestamp: P: 1781639920125759 usec, L: 0
0616 19:58:40.145748 (+ 717us) op_driver.cc:348] REPLICATION: starting
0616 19:58:40.298911 (+153163us) log.cc:844] Serialized 540 byte log entry
0616 19:58:40.566648 (+267737us) op_driver.cc:464] REPLICATION: finished
0616 19:58:40.641292 (+ 74644us) write_op.cc:301] APPLY: starting
0616 19:58:40.641406 (+ 114us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:40.641566 (+ 160us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:40.641738 (+ 172us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:40.642032 (+ 294us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:40.642261 (+ 229us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:40.642326 (+ 65us) write_op.cc:312] APPLY: finished
0616 19:58:40.642593 (+ 267us) log.cc:844] Serialized 22 byte log entry
0616 19:58:40.642917 (+ 324us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:40.643036 (+ 119us) write_op.cc:454] Released schema lock
0616 19:58:40.643090 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:40.643908 (+ 818us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:40.643956 (+ 48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":124587,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":74383,"apply.run_cpu_time_us":2115,"apply.run_wall_time_us":2611,"num_ops":1,"prepare.queue_time_us":166654,"prepare.run_cpu_time_us":13023,"prepare.run_wall_time_us":244539,"raft.queue_time_us":204175,"raft.run_cpu_time_us":15741,"raft.run_wall_time_us":75790,"replication_time_us":420757,"thread_start_us":398088,"threads_started":4,"wal-append.queue_time_us":53782}]]}
I20260616 19:58:40.809218 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:40.809844 8041 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:40.815567 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:40.816538 8041 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:40.820570 8043 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260616 19:58:40.820621 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:40.822443 8037 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:40.833756 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 100156us
I20260616 19:58:40.811172 8039 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.855974 8037 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:40.872256 9917 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260616 19:58:40.873813 10006 log.cc:927] Time spent T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.179s user 0.001s sys 0.000s
I20260616 19:58:40.875069 9917 raft_consensus.cc:993] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:40.875543 9917 raft_consensus.cc:1081] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:40.905997 9885 raft_consensus.cc:993] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:40.911124 9885 raft_consensus.cc:1081] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:40.911175 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:40.914106 8820 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.931874 9917 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.933393 8817 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.958444 8041 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:40.959084 8041 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:40.959444 8041 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:40.960881 8045 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:40.952713 9932 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:58:40.983419 7967 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:40.986009 7967 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:40.986382 7967 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:40.988754 8041 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:40.996734 7866 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:41.020846 7967 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:41.023542 9918 raft_consensus.cc:993] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:41.029631 7966 raft_consensus.cc:1275] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:41.029960 7967 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:41.032876 7866 catalog_manager.cc:5696] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.040442 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:41.041080 8112 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:41.049152 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:41.049806 8040 raft_consensus.cc:3055] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:41.050130 8040 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:41.053705 9861 consensus_queue.cc:1048] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:41.050848 8040 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:41.057368 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:41.057883 8040 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:41.057972 7968 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:41.066962 8041 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:41.068245 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:41.068912 8117 raft_consensus.cc:3055] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:41.068459 7967 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.070626 8041 raft_consensus.cc:1240] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:41.081317 8040 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:41.069242 8117 raft_consensus.cc:740] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:41.086601 8117 consensus_queue.cc:260] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:41.087400 7968 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:41.073035 9918 raft_consensus.cc:1081] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:41.094586 8117 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:41.097407 8112 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:41.111855 7934 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:41.117779 8043 raft_consensus.cc:1275] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:41.119342 9864 consensus_queue.cc:1048] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:41.129357 8117 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:41.124840 9970 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:41.135545 8011 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:41.136447 9377 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:41.136904 9377 raft_consensus.cc:697] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:41.137724 8045 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:58:41.138334 8045 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:41.138630 8045 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:41.144345 8045 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:41.149520 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:41.149632 7967 raft_consensus.cc:1275] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:41.150024 7966 raft_consensus.cc:3055] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:41.150240 7966 raft_consensus.cc:740] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:41.142997 9377 consensus_queue.cc:237] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:41.155189 7966 consensus_queue.cc:260] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:41.143323 9970 raft_consensus.cc:697] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:41.171254 7969 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:41.156452 9949 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260616 19:58:41.183130 9963 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
W20260616 19:58:41.207180 10016 log.cc:927] Time spent T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260616 19:58:41.188665 9970 consensus_queue.cc:237] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:41.218792 10017 log.cc:927] Time spent T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
I20260616 19:58:41.248075 8044 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.261387 8045 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:41.265205 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:41.265832 8117 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:41.266446 8112 raft_consensus.cc:1275] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:41.276820 9962 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:41.282642 8117 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:41.284840 7966 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:41.289712 8009 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:41.290649 9934 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:41.291150 9934 raft_consensus.cc:697] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:41.297109 7864 catalog_manager.cc:5696] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 3 to 4, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.306962 7865 catalog_manager.cc:5696] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:41.291986 9934 consensus_queue.cc:237] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:41.319326 8044 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:41.323607 7966 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:41.324622 9865 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260616 19:58:41.335821 7966 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:41.347350 7868 catalog_manager.cc:5696] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 5 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:41.359498 7868 catalog_manager.cc:5696] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:41.375994 8817 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:58:41.377621 9932 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:41.386170 9932 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.395536 8817 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.409662 8041 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:58:41.418526 8117 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:41.418780 7866 catalog_manager.cc:5696] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:41.426056 9910 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260616 19:58:41.432916 8820 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260616 19:58:41.433430 8044 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:58:41.435441 7855 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 0d19ee585ead4942a40096f7aef3ba50 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:41.447160 9861 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:41.457690 7966 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260616 19:58:41.459530 9861 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:41.462033 9861 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.464900 7864 catalog_manager.cc:5696] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 6, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.467463 8820 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.488785 8101 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d19ee585ead4942a40096f7aef3ba50 with delete_type TABLET_DATA_TOMBSTONED (TS 6827184e3cef42c2ada15b2d31afee4b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:41.491014 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet ba7d833edb374b10abaeb128d2084328 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260616 19:58:41.496634 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:41.500852 8117 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:41.504199 8041 raft_consensus.cc:2955] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.507519 10027 tablet_replica.cc:333] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: stopping tablet replica
I20260616 19:58:41.510259 7966 raft_consensus.cc:2955] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.515188 8817 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:58:41.519480 8045 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:58:41.520982 9861 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:41.522791 7865 catalog_manager.cc:5696] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:41.523494 9861 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:41.530682 7865 catalog_manager.cc:5696] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:41.533157 9917 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.538805 10027 raft_consensus.cc:2243] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:41.542987 10027 raft_consensus.cc:2272] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:41.543536 8045 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.546614 10027 ts_tablet_manager.cc:1916] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:41.553004 8817 raft_consensus.cc:2955] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.557842 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 43378ff312f94246811895976a8715d6 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:41.568699 7966 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:41.571099 10027 ts_tablet_manager.cc:1929] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260616 19:58:41.571595 10027 log.cc:1199] T 0d19ee585ead4942a40096f7aef3ba50 P 6827184e3cef42c2ada15b2d31afee4b: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root/wals/0d19ee585ead4942a40096f7aef3ba50
I20260616 19:58:41.571771 9948 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:41.573508 7853 catalog_manager.cc:5027] TS 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): tablet 0d19ee585ead4942a40096f7aef3ba50 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:41.577512 7868 catalog_manager.cc:5696] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index 4 to 5, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:41.650605 8037 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:41.663793 9918 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260616 19:58:41.743702 8028 tablet_service.cc:1558] Processing DeleteTablet for tablet ba7d833edb374b10abaeb128d2084328 with delete_type TABLET_DATA_TOMBSTONED (TS de4c5ce3ea8345ae9068f825b8d60e19 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:41.758491 10030 tablet_replica.cc:333] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: stopping tablet replica
I20260616 19:58:41.772202 10030 raft_consensus.cc:2243] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:41.777503 10030 raft_consensus.cc:2272] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:41.789662 7864 catalog_manager.cc:5696] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:41.799062 8117 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:41.815016 9946 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:41.834091 8117 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:41.834265 8114 raft_consensus.cc:1275] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:41.836892 10030 ts_tablet_manager.cc:1916] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:41.846410 9880 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:41.850229 9952 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:41.884033 7953 tablet_service.cc:1558] Processing DeleteTablet for tablet 43378ff312f94246811895976a8715d6 with delete_type TABLET_DATA_TOMBSTONED (TS b022cac8cb5e46b7a39a748e81e257a7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49182
W20260616 19:58:41.867744 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 194306us
W20260616 19:58:41.942406 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 71484.8us
I20260616 19:58:41.963604 10036 tablet_replica.cc:333] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: stopping tablet replica
I20260616 19:58:42.016083 7968 raft_consensus.cc:1275] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:42.091030 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:42.153759 9934 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:42.189200 10030 ts_tablet_manager.cc:1929] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260616 19:58:42.189760 10030 log.cc:1199] T ba7d833edb374b10abaeb128d2084328 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root/wals/ba7d833edb374b10abaeb128d2084328
I20260616 19:58:42.191475 7854 catalog_manager.cc:5027] TS de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): tablet ba7d833edb374b10abaeb128d2084328 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:42.208628 10036 raft_consensus.cc:2243] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:42.211268 10036 raft_consensus.cc:2272] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:42.214588 10036 ts_tablet_manager.cc:1916] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:42.256021 10036 ts_tablet_manager.cc:1929] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260616 19:58:42.293169 10036 log.cc:1199] T 43378ff312f94246811895976a8715d6 P b022cac8cb5e46b7a39a748e81e257a7: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root/wals/43378ff312f94246811895976a8715d6
I20260616 19:58:42.294828 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:42.295621 8112 raft_consensus.cc:606] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:42.296959 7855 catalog_manager.cc:5027] TS b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): tablet 43378ff312f94246811895976a8715d6 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:42.493927 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:42.495508 8112 raft_consensus.cc:606] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:42.523097 8037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:42.523726 8037 raft_consensus.cc:606] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:42.593495 7966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:42.594159 7966 raft_consensus.cc:606] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:42.600221 7966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:42.600951 7966 raft_consensus.cc:606] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:42.655442 8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:42.656530 8117 raft_consensus.cc:606] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Received request to transfer leadership to TS c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:42.669819 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260616 19:58:42.671584 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:42.672693 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:42.680614 9934 raft_consensus.cc:993] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:42.696344 9934 raft_consensus.cc:1081] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:42.738351 7963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:42.739008 7963 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:42.739416 7963 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:42.749225 7963 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.752379 7963 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:42.755439 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:42.768975 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:42.781764 8043 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:42.781692 8115 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:42.782167 8043 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:42.790638 8043 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.791898 8043 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:42.792688 9996 raft_consensus.cc:993] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:42.793175 9996 raft_consensus.cc:1081] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:42.797875 8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:42.798508 8117 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:42.798871 8117 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:42.811252 10045 raft_consensus.cc:993] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:42.811815 10045 raft_consensus.cc:1081] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Signalling peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:42.817461 8117 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.822862 8043 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:42.885139 8117 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:42.888978 8817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328"
dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:52458
I20260616 19:58:42.889647 8817 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:42.890021 8817 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:42.895248 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:42.897909 7967 raft_consensus.cc:3055] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:42.897413 10045 raft_consensus.cc:993] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:42.898569 10045 raft_consensus.cc:1081] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:42.897073 9893 raft_consensus.cc:993] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:42.899757 9893 raft_consensus.cc:1081] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:42.900637 7967 raft_consensus.cc:740] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:42.901590 7967 consensus_queue.cc:260] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.902312 8115 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:42.902904 7967 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:42.904253 7934 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:42.907140 9928 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:42.907923 9928 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:42.909024 9928 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.912669 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:42.933151 8041 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:42.915774 7966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:42.935209 7966 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:42.935653 7966 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:42.916821 7969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:42.936407 7969 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:42.936751 7969 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:42.941359 10047 raft_consensus.cc:993] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:42.941897 10047 raft_consensus.cc:1081] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:42.948207 8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:42.948796 8115 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:42.949158 8115 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:42.954278 7966 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.920779 8817 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:42.985680 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:42.986377 7961 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:42.986583 8817 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:42.958586 7969 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.970517 8115 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.989483 8115 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:42.989959 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:42.990522 8117 raft_consensus.cc:3055] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:42.990832 8117 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:42.991952 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:42.992491 8045 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:42.992825 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:42.990705 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:42.993356 8044 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:42.993481 8115 raft_consensus.cc:3055] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:42.993808 8115 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:42.993886 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:42.991595 8117 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:42.994532 8043 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:42.995071 8117 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:42.994727 8115 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:42.995543 8041 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:42.995823 8115 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:42.996548 7969 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:42.996706 8084 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:42.997862 9893 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:42.998317 9893 raft_consensus.cc:697] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:42.999207 9893 consensus_queue.cc:237] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:43.002455 7966 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:43.003459 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:43.004053 8116 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:43.004376 8116 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:43.005185 8116 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:43.006291 8116 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:43.007244 7967 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:43.017006 7868 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 7, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:43.021418 7961 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 3.
I20260616 19:58:43.022145 8043 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
W20260616 19:58:42.979274 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49664.3us
I20260616 19:58:43.022629 8786 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:58:43.023721 10033 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:43.024981 7935 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:43.028347 9928 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:43.028853 9928 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:43.030366 9928 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:43.034126 8044 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:43.034072 8045 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:43.035661 8084 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:43.036957 9864 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:43.037292 7935 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:43.037900 9864 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:43.038149 10047 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:43.038684 10047 raft_consensus.cc:697] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:43.039115 8117 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 3.
I20260616 19:58:43.038938 9864 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:43.039584 10047 consensus_queue.cc:237] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:43.037200 10033 raft_consensus.cc:697] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:58:43.048372 10033 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:43.082101 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:43.082903 7966 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:43.083325 7966 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:43.084255 7966 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:43.085846 7966 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:43.106865 8115 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:43.116554 8116 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:43.127372 7966 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:43.136934 7967 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:43.152364 7961 raft_consensus.cc:1240] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:43.185645 7864 catalog_manager.cc:5696] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 2 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 3 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:43.191273 7866 catalog_manager.cc:5696] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:43.279999 7865 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:43.282660 7865 catalog_manager.cc:5696] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:43.299472 7864 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 6 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
W20260616 19:58:43.384896 7916 auto_rebalancer.cc:591] Failed to schedule move for tablet 3dc4c19625e84ce29cb4ca6993524bf1: Illegal state: Leader has not yet committed an operation in its own term
I20260616 19:58:43.390240 8043 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:58:43.397548 8008 consensus_peers.cc:597] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33c17f5e21da4e9893d22581a43ceafb. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:43.397732 7966 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:43.399067 8116 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:43.399523 9875 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:43.400724 9885 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:43.416471 8115 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:43.415524 9946 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:43.435698 7961 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:43.441255 8043 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:58:43.465257 8008 consensus_peers.cc:597] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 854c79c08f4d4f61a0e1ff9f6c5ad2c0. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:43.481657 7864 catalog_manager.cc:5696] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:43.483845 7966 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260616 19:58:43.488049 8117 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260616 19:58:43.491849 9988 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:43.500289 8115 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:43.503137 9934 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:43.504590 10047 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:43.501623 8045 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:43.512529 7966 raft_consensus.cc:1275] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:43.515012 9917 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:43.518440 8117 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:43.520130 9952 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:43.521458 9928 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:43.535271 7967 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:43.544020 9864 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:43.547407 8044 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:43.551275 9917 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:43.559757 8116 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260616 19:58:43.563632 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08002e12a0 after lost signal to thread 6785
I20260616 19:58:43.563712 9928 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260616 19:58:43.565032 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08002de500 after lost signal to thread 7902
W20260616 19:58:43.566322 8140 debug-util.cc:398] Leaking SignalData structure 0x7b0800085d20 after lost signal to thread 7994
W20260616 19:58:43.567427 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08002e19a0 after lost signal to thread 8069
W20260616 19:58:43.568564 8140 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb6e0 after lost signal to thread 8143
W20260616 19:58:43.569753 8140 debug-util.cc:398] Leaking SignalData structure 0x7b0800218a60 after lost signal to thread 8845
I20260616 19:58:43.577133 8045 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:43.581966 7966 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260616 19:58:43.583360 10033 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:43.594760 8045 raft_consensus.cc:1275] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:43.596361 8043 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:43.608930 10047 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:43.610764 10047 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:43.615558 10048 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:43.798930 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:43.832824 7961 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:43.845530 8115 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260616 19:58:43.901515 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 211139us
W20260616 19:58:43.983626 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 74605.4us
W20260616 19:58:43.987202 10094 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.195s user 0.001s sys 0.000s
W20260616 19:58:43.949713 10052 log.cc:927] Time spent T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260616 19:58:43.911702 10033 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
W20260616 19:58:43.998009 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 84570.5us
I20260616 19:58:44.069921 8110 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:43.900043 9875 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
W20260616 19:58:44.090045 10097 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
W20260616 19:58:44.117317 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 57091.5us
W20260616 19:58:44.117811 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 236086us
I20260616 19:58:44.166472 8040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:44.167116 8040 raft_consensus.cc:606] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:44.176910 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 214462us
I20260616 19:58:44.187434 8039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
W20260616 19:58:44.225597 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 168228us
I20260616 19:58:44.237412 10109 raft_consensus.cc:993] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:44.245908 10109 raft_consensus.cc:1081] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:44.257789 7864 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index -1 to 4, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:44.260972 8039 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:44.268922 10121 raft_consensus.cc:993] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:44.270087 10127 ts_tablet_manager.cc:933] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:44.271104 7966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:44.272574 7966 raft_consensus.cc:606] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:44.272095 10121 raft_consensus.cc:1081] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:44.275563 7966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:44.276377 7966 raft_consensus.cc:606] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:44.277845 10128 ts_tablet_manager.cc:933] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:44.280747 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:44.281334 8110 raft_consensus.cc:606] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:58:44.282960 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 230070us
I20260616 19:58:44.283210 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:44.285000 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:44.285813 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:44.295825 7961 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:44.296434 7961 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:44.296808 7961 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:44.297281 7963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:44.297824 7963 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:44.298167 7963 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:44.316319 10119 raft_consensus.cc:993] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:44.317010 10119 raft_consensus.cc:1081] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:44.316370 7968 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:58:44.323771 7933 consensus_peers.cc:597] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00bf27a79bc04305bb057b972a08c316. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:44.326555 7963 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:44.329085 7963 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:44.334604 8044 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260616 19:58:44.332947 7961 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.336462 8117 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260616 19:58:44.346544 10119 raft_consensus.cc:993] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:44.347103 10119 raft_consensus.cc:1081] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:44.335484 8114 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:44.349433 8114 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:44.349785 8114 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:44.346408 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:44.352089 8043 raft_consensus.cc:3055] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:44.354408 10083 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:44.355016 8043 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:44.355852 8043 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:44.357334 8043 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:44.357381 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:44.358011 8115 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:44.366652 8043 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:44.368046 8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:44.369652 8117 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:44.370007 8117 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:44.371071 8115 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:44.372650 7935 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:44.379928 7966 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:58:44.381675 10116 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:44.382246 9885 consensus_queue.cc:1059] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:44.383234 9885 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:44.383527 9885 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:44.382489 10116 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:44.384395 9885 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.372784 8114 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.388733 7961 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:44.391566 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:44.385253 9885 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:44.400287 9918 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260616 19:58:44.404181 8114 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:44.412170 8117 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.416257 8117 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:44.417032 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:44.422240 7963 raft_consensus.cc:3055] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:44.423791 7963 raft_consensus.cc:740] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:44.425223 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:44.424752 7963 consensus_queue.cc:260] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.425834 8044 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:44.426208 7963 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:44.427559 8043 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:44.384608 10116 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:44.417496 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:44.428647 7961 raft_consensus.cc:3055] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:44.431991 7961 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:44.433059 7961 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.434763 10128 tablet_copy_client.cc:323] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.130:43051
I20260616 19:58:44.434384 7961 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:44.436982 8055 tablet_copy_service.cc:140] P de4c5ce3ea8345ae9068f825b8d60e19: Received BeginTabletCopySession request for tablet 33c17f5e21da4e9893d22581a43ceafb from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:46718)
I20260616 19:58:44.437552 8055 tablet_copy_service.cc:161] P de4c5ce3ea8345ae9068f825b8d60e19: Beginning new tablet copy session on tablet 33c17f5e21da4e9893d22581a43ceafb from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:46718: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-33c17f5e21da4e9893d22581a43ceafb
I20260616 19:58:44.442785 8044 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:44.446311 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:44.446945 8117 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:44.439272 9918 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:44.444972 8040 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:44.455124 8115 raft_consensus.cc:1240] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 1. Current term is 2. Ops: []
I20260616 19:58:44.457777 8113 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:44.457672 7963 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:44.459913 7961 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:44.468288 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:44.485353 8038 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:44.487473 8055 tablet_copy_source_session.cc:215] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: Tablet Copy: opened 0 blocks and 1 log segments
W20260616 19:58:44.490295 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:44.493232 7935 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:44.490811 8085 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:44.494797 8085 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:44.499269 8038 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:58:44.502526 8117 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:44.500494 10080 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:44.497311 10116 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:44.513681 10080 raft_consensus.cc:697] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:44.515771 10116 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:44.521340 10101 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:44.553129 10101 raft_consensus.cc:697] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:44.554054 10101 consensus_queue.cc:237] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.525209 10128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33c17f5e21da4e9893d22581a43ceafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:44.539086 10080 consensus_queue.cc:237] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.547430 10116 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.587932 10088 raft_consensus.cc:993] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:44.589205 10088 raft_consensus.cc:1081] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:44.591444 7961 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:44.592049 7961 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:44.592483 7961 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:44.612354 7961 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.615280 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:44.615890 8043 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:44.521311 10127 tablet_copy_client.cc:323] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.130:43051
I20260616 19:58:44.622859 8055 tablet_copy_service.cc:140] P de4c5ce3ea8345ae9068f825b8d60e19: Received BeginTabletCopySession request for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:46718)
I20260616 19:58:44.623430 8055 tablet_copy_service.cc:161] P de4c5ce3ea8345ae9068f825b8d60e19: Beginning new tablet copy session on tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:46718: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-854c79c08f4d4f61a0e1ff9f6c5ad2c0
I20260616 19:58:44.624441 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:44.625046 8116 raft_consensus.cc:3055] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:44.625342 8116 raft_consensus.cc:740] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:44.626190 8116 consensus_queue.cc:260] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.627517 8116 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:44.631567 8055 tablet_copy_source_session.cc:215] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:44.637856 10127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:44.633693 8043 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:44.641667 7935 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:44.615422 7961 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:44.632709 7866 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:44.648258 8116 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:44.676834 10126 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:44.687110 10126 raft_consensus.cc:697] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:44.714634 10126 consensus_queue.cc:237] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:44.719893 7865 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 4, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:44.732563 7865 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 7, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:44.763849 7866 catalog_manager.cc:5696] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 1 to 2, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 2 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:44.767575 7866 catalog_manager.cc:5696] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 5 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:44.789608 7961 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49182
I20260616 19:58:44.773885 10128 tablet_copy_client.cc:806] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:44.790458 7961 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Received request to transfer leadership
I20260616 19:58:44.815286 10127 tablet_copy_client.cc:806] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:44.815949 10127 tablet_copy_client.cc:670] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:44.824134 10127 tablet_copy_client.cc:538] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:44.827234 7865 catalog_manager.cc:5696] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:44.793841 10128 tablet_copy_client.cc:670] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:44.836201 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49182
I20260616 19:58:44.836830 7963 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Received request to transfer leadership
W20260616 19:58:44.841979 7933 consensus_peers.cc:597] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260616 19:58:44.842689 7855 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 854c79c08f4d4f61a0e1ff9f6c5ad2c0 already in progress [suppressed 3 similar messages]
I20260616 19:58:44.853760 10142 ts_tablet_manager.cc:933] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:58:44.856554 10128 tablet_copy_client.cc:538] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:44.861985 7961 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49182
I20260616 19:58:44.862576 7961 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Received request to transfer leadership
I20260616 19:58:44.864406 10127 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
W20260616 19:58:44.868091 7916 auto_rebalancer.cc:855] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260616 19:58:44.868546 7916 auto_rebalancer.cc:863] Could not move replica: Remote error: Service unavailable: leadership transfer for 854c79c08f4d4f61a0e1ff9f6c5ad2c0 already in progress
W20260616 19:58:44.868810 7916 auto_rebalancer.cc:301] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 854c79c08f4d4f61a0e1ff9f6c5ad2c0 already in progress
I20260616 19:58:44.870604 10128 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:44.878259 10142 tablet_copy_client.cc:323] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
I20260616 19:58:44.880272 7980 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet 00bf27a79bc04305bb057b972a08c316 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
I20260616 19:58:44.880816 7980 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet 00bf27a79bc04305bb057b972a08c316 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-00bf27a79bc04305bb057b972a08c316
I20260616 19:58:44.892081 7980 tablet_copy_source_session.cc:215] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:44.895926 10142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00bf27a79bc04305bb057b972a08c316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:44.942718 10142 tablet_copy_client.cc:806] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:44.944638 10142 tablet_copy_client.cc:670] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:44.954597 10127 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:44.954588 10142 tablet_copy_client.cc:538] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:44.956030 10127 tablet_bootstrap.cc:492] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:44.958745 10127 ts_tablet_manager.cc:1403] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.098s user 0.023s sys 0.009s
I20260616 19:58:44.970206 10127 raft_consensus.cc:359] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:44.971115 10127 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:44.971331 7961 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:44.971786 10127 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:44.973126 10080 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:44.979374 10127 ts_tablet_manager.cc:1434] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260616 19:58:44.983412 8055 tablet_copy_service.cc:342] P de4c5ce3ea8345ae9068f825b8d60e19: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-854c79c08f4d4f61a0e1ff9f6c5ad2c0 received from {username='slave'} at 127.0.0.1:46718
I20260616 19:58:44.983917 8055 tablet_copy_service.cc:434] P de4c5ce3ea8345ae9068f825b8d60e19: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-854c79c08f4d4f61a0e1ff9f6c5ad2c0 on tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:44.999274 8117 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260616 19:58:45.002720 10126 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:45.008415 8116 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:45.014199 10126 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:45.015691 10116 raft_consensus.cc:993] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:45.016227 10116 raft_consensus.cc:1081] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:45.018232 8114 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:45.021211 8114 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:45.021704 8114 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:45.035168 10142 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:45.061693 8114 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:45.086329 8044 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260616 19:58:45.094059 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:45.107266 8044 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:45.109104 8038 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:58:45.120384 7968 raft_consensus.cc:3055] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:45.120839 7968 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:45.122315 8114 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:45.128721 10102 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260616 19:58:45.128548 10048 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260616 19:58:45.140903 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:45.141621 8044 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:45.141314 7968 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:45.142848 7968 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:45.151238 8044 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:45.175781 8113 raft_consensus.cc:1275] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:45.187659 10126 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:58:45.191066 10142 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:45.193120 10142 tablet_bootstrap.cc:492] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:45.195055 10142 ts_tablet_manager.cc:1403] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.160s user 0.037s sys 0.002s
I20260616 19:58:45.203577 7968 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:45.210765 8085 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:45.212703 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 84853.1us
I20260616 19:58:45.220854 10048 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:45.221428 10048 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:45.221287 10142 raft_consensus.cc:359] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:45.222255 10142 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
W20260616 19:58:45.213160 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 83929.6us
I20260616 19:58:45.222420 10048 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:45.223027 10142 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:45.228715 8038 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:45.254601 7961 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:45.258473 10048 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:45.267086 8038 raft_consensus.cc:1275] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:45.276851 10142 ts_tablet_manager.cc:1434] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.077s user 0.008s sys 0.000s
I20260616 19:58:45.280491 10116 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:45.284329 10120 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:58:45.300770 7980 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-00bf27a79bc04305bb057b972a08c316 received from {username='slave'} at 127.0.0.1:37398
I20260616 19:58:45.301263 7980 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-00bf27a79bc04305bb057b972a08c316 on tablet 00bf27a79bc04305bb057b972a08c316 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:45.315480 10128 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:45.330446 10128 tablet_bootstrap.cc:492] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
W20260616 19:58:45.316833 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 93212.8us
I20260616 19:58:45.337050 10128 ts_tablet_manager.cc:1403] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.467s user 0.071s sys 0.040s
I20260616 19:58:45.326886 7867 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:45.341018 10128 raft_consensus.cc:359] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:45.352144 10128 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:45.354226 10128 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:45.373852 10128 ts_tablet_manager.cc:1434] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260616 19:58:45.377942 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:45.401710 8055 tablet_copy_service.cc:342] P de4c5ce3ea8345ae9068f825b8d60e19: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-33c17f5e21da4e9893d22581a43ceafb received from {username='slave'} at 127.0.0.1:46718
I20260616 19:58:45.402206 8055 tablet_copy_service.cc:434] P de4c5ce3ea8345ae9068f825b8d60e19: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-33c17f5e21da4e9893d22581a43ceafb on tablet 33c17f5e21da4e9893d22581a43ceafb with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:45.592666 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:45.593283 8044 raft_consensus.cc:606] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:45.598184 7969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:45.598783 7969 raft_consensus.cc:606] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:45.601809 8113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:45.602384 8113 raft_consensus.cc:606] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:45.603951 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260616 19:58:45.608660 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:45.613279 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:45.691635 8039 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:45.693011 10153 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:45.716375 10102 raft_consensus.cc:993] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:45.716921 10102 raft_consensus.cc:1081] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:45.718497 8044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:45.719005 8044 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:45.719339 8044 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:45.746487 8820 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEARNER]: Advancing to term 5
I20260616 19:58:45.747910 7961 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:45.756036 8044 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:45.758206 8044 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:45.756038 8820 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:45.761495 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:45.762089 7969 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:45.756045 8819 raft_consensus.cc:1217] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Deduplicated request from leader. Original: 5.3->[5.4-5.4] Dedup: 5.4->[]
I20260616 19:58:45.769294 10153 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:45.776140 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:45.776796 8113 raft_consensus.cc:3055] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:45.777119 8113 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:45.777787 8113 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:45.778585 8113 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:45.776090 10101 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260616 19:58:45.802786 10116 raft_consensus.cc:993] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:45.803356 10116 raft_consensus.cc:1081] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:45.807548 8114 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:45.808096 8114 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:45.809657 8114 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:45.788008 8113 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:45.828867 8009 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:45.829945 9951 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:45.830528 9951 raft_consensus.cc:697] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:45.831969 9951 consensus_queue.cc:237] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:45.835187 8817 raft_consensus.cc:1217] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEARNER]: Deduplicated request from leader. Original: 6.5->[6.6-6.6] Dedup: 6.6->[]
I20260616 19:58:45.845248 7969 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:58:45.845940 7961 raft_consensus.cc:1240] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:45.847152 8114 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:45.853204 8114 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:45.854773 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:45.854965 7865 catalog_manager.cc:5696] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 2 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 3 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:45.855365 7961 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:45.865603 7961 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:45.869634 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:45.870229 8043 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:45.870719 7961 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:45.871997 7961 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:45.878981 8043 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:45.881325 8084 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:45.882170 10153 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:45.882754 10153 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:45.883785 10153 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:45.898613 7961 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:45.940868 10048 raft_consensus.cc:1064] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:45.954636 10048 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:45.974323 8819 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260616 19:58:45.981052 7961 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260616 19:58:45.982834 10048 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:58:45.994544 10102 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260616 19:58:46.033998 10168 log.cc:927] Time spent T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
I20260616 19:58:46.038002 10048 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 4 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.074894 7969 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 4 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.088459 10085 raft_consensus.cc:993] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:46.088974 10085 raft_consensus.cc:1081] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:46.090977 7961 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:46.091593 7961 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:46.091970 7961 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:46.054240 8819 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 4 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.109414 8043 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260616 19:58:46.148129 7864 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:46.149770 7961 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:46.173794 9893 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:46.186033 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:46.187249 8114 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:46.188059 7961 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:46.194669 7864 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 4 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.216563 8044 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 4 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.227521 8114 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:46.229729 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:46.230307 8043 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:46.230638 8043 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:46.231353 8043 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:46.232646 8043 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:46.233953 7934 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:46.235289 10083 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:46.235780 10083 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:46.236610 10083 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:46.246704 8043 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:58:46.316869 7969 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:46.331332 10102 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:46.337644 8044 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:46.339401 10102 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:46.351203 9951 raft_consensus.cc:1064] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:46.354063 9951 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 2, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:46.371803 8113 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:46.376995 10155 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:46.411346 10116 raft_consensus.cc:1064] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:46.426546 10116 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:46.514904 7864 catalog_manager.cc:5696] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:46.616284 8819 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:58:46.618290 8114 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:58:46.618462 10155 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:46.626327 10155 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260616 19:58:46.638187 8820 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:46.644616 10083 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:46.674945 10085 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.678525 7968 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:46.692754 8114 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.693866 7968 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:58:46.693147 8819 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.720963 10184 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:46.702036 7865 catalog_manager.cc:5696] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index 4 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260616 19:58:46.703681 10185 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
I20260616 19:58:46.714733 8044 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260616 19:58:46.720244 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 118538us
I20260616 19:58:46.740830 8117 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:46.758288 8116 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:46.761435 8044 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:46.766855 8819 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:46.768059 9951 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:46.768477 8114 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:46.770386 8114 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:46.776015 10085 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.777894 9951 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:46.777766 8819 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.790673 8115 raft_consensus.cc:2955] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.792660 7854 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 33c17f5e21da4e9893d22581a43ceafb with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:46.804214 8817 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260616 19:58:46.805368 8044 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260616 19:58:46.806556 10126 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:46.807399 9893 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:58:46.808280 10048 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:58:46.813899 7865 catalog_manager.cc:5696] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 6, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.818424 10153 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Committing config change with OpId 5.8: config changed from index 7 to 8, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.825783 8044 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Committing config change with OpId 5.8: config changed from index 7 to 8, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.828688 8819 raft_consensus.cc:2955] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Committing config change with OpId 5.8: config changed from index 7 to 8, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.839778 10116 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:46.842294 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:46.874740 7865 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index 7 to 8, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:46.882004 8040 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:46.893599 7953 tablet_service.cc:1558] Processing DeleteTablet for tablet 33c17f5e21da4e9893d22581a43ceafb with delete_type TABLET_DATA_TOMBSTONED (TS b022cac8cb5e46b7a39a748e81e257a7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49182
I20260616 19:58:46.914222 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:46.950847 7958 tablet_service.cc:1558] Processing DeleteTablet for tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 with delete_type TABLET_DATA_TOMBSTONED (TS b022cac8cb5e46b7a39a748e81e257a7 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:49182
I20260616 19:58:46.951301 10192 tablet_replica.cc:333] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: stopping tablet replica
I20260616 19:58:46.952777 10192 raft_consensus.cc:2243] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:46.955957 8116 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.955708 10116 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:46.991199 10192 raft_consensus.cc:2272] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260616 19:58:47.001962 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 86798.4us
W20260616 19:58:47.002574 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45498.8us
W20260616 19:58:47.002779 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 95646.1us
I20260616 19:58:47.003677 10182 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:58:47.007524 10192 ts_tablet_manager.cc:1916] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:47.031067 10172 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:47.073810 7864 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:47.075732 8044 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:47.079874 8819 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:47.083482 8043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:47.086133 8043 raft_consensus.cc:606] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:47.101320 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:47.101920 8115 raft_consensus.cc:606] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:47.108948 7968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:47.109593 7968 raft_consensus.cc:606] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:47.113593 10192 ts_tablet_manager.cc:1929] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.4
I20260616 19:58:47.114089 10192 log.cc:1199] T 33c17f5e21da4e9893d22581a43ceafb P b022cac8cb5e46b7a39a748e81e257a7: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root/wals/33c17f5e21da4e9893d22581a43ceafb
I20260616 19:58:47.114809 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:47.115754 8116 raft_consensus.cc:606] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:47.119881 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260616 19:58:47.120857 10192 tablet_replica.cc:333] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: stopping tablet replica
I20260616 19:58:47.122030 10192 raft_consensus.cc:2243] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:47.123016 10192 raft_consensus.cc:2272] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:47.115653 7855 catalog_manager.cc:5027] TS b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): tablet 33c17f5e21da4e9893d22581a43ceafb (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:47.143977 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:47.144831 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:47.160655 9893 raft_consensus.cc:993] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:47.161173 10153 raft_consensus.cc:1081] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:47.163250 10192 ts_tablet_manager.cc:1916] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:47.163818 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:47.164398 7968 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:47.164829 7968 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:47.183329 7968 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.185205 7968 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:47.186275 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:47.186901 8116 raft_consensus.cc:3055] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:47.187237 8116 raft_consensus.cc:740] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:47.187989 8116 consensus_queue.cc:260] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.189256 8116 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:47.195514 7969 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:47.201229 8819 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260616 19:58:47.203423 10193 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:58:47.205017 8115 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260616 19:58:47.207497 10193 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:58:47.216715 10116 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:47.219087 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:47.219704 8038 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:47.228845 7855 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 00bf27a79bc04305bb057b972a08c316 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:47.230688 8115 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:47.235699 8819 raft_consensus.cc:2955] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:47.258267 8038 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:47.258074 7867 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index 7 to 8, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:47.272899 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:47.273689 10193 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:47.275333 10193 raft_consensus.cc:697] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:47.276270 10193 consensus_queue.cc:237] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.280112 8028 tablet_service.cc:1558] Processing DeleteTablet for tablet 00bf27a79bc04305bb057b972a08c316 with delete_type TABLET_DATA_TOMBSTONED (TS de4c5ce3ea8345ae9068f825b8d60e19 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:47.293229 10201 tablet_replica.cc:333] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: stopping tablet replica
I20260616 19:58:47.294412 10201 raft_consensus.cc:2243] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:47.295215 10201 raft_consensus.cc:2272] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:47.301592 10201 ts_tablet_manager.cc:1916] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:47.247819 8116 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:47.401705 10192 ts_tablet_manager.cc:1929] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.7
I20260616 19:58:47.416641 10192 log.cc:1199] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P b022cac8cb5e46b7a39a748e81e257a7: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root/wals/854c79c08f4d4f61a0e1ff9f6c5ad2c0
I20260616 19:58:47.418344 7855 catalog_manager.cc:5027] TS b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): tablet 854c79c08f4d4f61a0e1ff9f6c5ad2c0 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:47.427699 7865 catalog_manager.cc:5696] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:47.440680 10085 raft_consensus.cc:993] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:47.441191 10085 raft_consensus.cc:1081] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:47.443531 10193 raft_consensus.cc:993] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:47.444024 10193 raft_consensus.cc:1081] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:47.445989 8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:47.446532 8115 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:47.446879 8115 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:47.458634 10201 ts_tablet_manager.cc:1929] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.7
I20260616 19:58:47.459124 10201 log.cc:1199] T 00bf27a79bc04305bb057b972a08c316 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root/wals/00bf27a79bc04305bb057b972a08c316
I20260616 19:58:47.460520 8115 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.462491 8115 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:47.463685 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:47.464372 8044 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:47.480914 7854 catalog_manager.cc:5027] TS de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): tablet 00bf27a79bc04305bb057b972a08c316 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:47.485329 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:47.485646 8044 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:47.485977 7961 raft_consensus.cc:3055] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:47.486297 7961 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:47.486784 8084 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:47.487095 7961 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.487676 10194 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:47.488204 10194 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:47.488855 7961 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:47.489034 10194 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.510782 7961 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
W20260616 19:58:47.548774 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 124751us
I20260616 19:58:47.552358 7961 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:47.552953 7961 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:47.553311 7961 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:47.573601 7961 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.576800 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:47.582517 8044 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:47.587057 8044 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:47.588277 8044 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.589758 8044 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:58:47.596664 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 77474.4us
I20260616 19:58:47.601130 7865 catalog_manager.cc:5696] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 6 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:47.674978 7961 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:47.679911 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:47.687006 8115 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:47.703239 8114 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:47.705372 10116 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260616 19:58:47.732630 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49429.6us
I20260616 19:58:47.737255 10195 raft_consensus.cc:993] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:47.756678 10207 raft_consensus.cc:1081] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:47.700330 8115 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:47.762761 7934 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:58:47.761595 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:47.768738 8038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:47.769439 8038 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:47.773358 8038 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:47.773902 10196 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:47.798959 10196 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:47.799715 8044 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:58:47.799912 10196 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.813995 8044 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:47.817694 10172 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:47.828598 8038 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.832085 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:47.832724 8116 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:47.833022 8116 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:47.833806 8116 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.834954 8116 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:47.848035 8116 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:47.848860 8038 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:47.849246 8009 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:47.850244 10203 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
W20260616 19:58:47.853248 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 47971.3us
I20260616 19:58:47.856278 10203 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:47.867054 10203 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:47.889869 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:47.890549 7969 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:47.898051 7969 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:58:47.898172 7867 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 5 to 6, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:47.906003 7865 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:48.082425 7963 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:48.084051 10153 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:48.095814 8044 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:48.100566 10194 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:48.251309 7963 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:48.253870 10203 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:48.321218 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:48.339708 8115 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:58:48.342675 10203 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:48.419912 8114 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:48.426198 10196 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:48.454332 8044 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:48.480762 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:48.481511 8044 raft_consensus.cc:606] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:48.488659 8114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:48.490307 8114 raft_consensus.cc:606] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:48.499343 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:48.499984 8044 raft_consensus.cc:606] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:48.503091 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8890d947e59455187876dccb01c8c06"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:48.505944 7963 raft_consensus.cc:606] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:48.512833 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:48.513562 7963 raft_consensus.cc:606] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:48.514956 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:48.516837 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:48.517704 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260616 19:58:48.525105 7935 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.6.159.190:44129, user_credentials={real_user=slave}} blocked reactor thread for 46666.6us
I20260616 19:58:48.579711 10172 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:48.610903 7963 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:48.642652 10172 raft_consensus.cc:993] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:48.661144 10196 raft_consensus.cc:1081] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:48.663233 8116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8890d947e59455187876dccb01c8c06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:48.664110 8116 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:48.665383 8116 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:58:48.691800 7933 consensus_peers.cc:597] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e92f5df5ec84f069decf29684f08b06. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:48.702987 8111 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:58:48.704403 10219 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:48.705030 10207 raft_consensus.cc:993] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:48.705567 10153 raft_consensus.cc:1081] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:48.707326 8819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269"
dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:52458
I20260616 19:58:48.707836 8819 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:48.708272 8819 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:48.711899 8116 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.714323 8116 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:48.715052 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:48.718834 7961 raft_consensus.cc:3055] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:48.719165 7961 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:48.719852 7961 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.720683 7961 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:48.718490 10219 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:48.715579 8819 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:48.717201 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:48.728896 8038 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:48.731942 8819 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:48.731873 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:48.732457 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:48.732944 7969 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:48.733031 8111 raft_consensus.cc:3055] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:48.733428 8111 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:48.734269 7961 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:48.734123 8111 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:48.735421 8085 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:48.736251 10207 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:48.736948 10207 raft_consensus.cc:697] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:48.737774 10207 consensus_queue.cc:237] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.739758 8111 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:48.741174 7969 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 2.
I20260616 19:58:48.742580 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:58:48.739049 8116 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:48.743534 10188 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:48.754408 8038 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:58:48.759060 10188 raft_consensus.cc:697] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:58:48.760911 10188 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:48.762072 8111 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 2.
W20260616 19:58:48.717669 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 100280us
I20260616 19:58:48.763779 10116 raft_consensus.cc:993] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:48.764245 10221 raft_consensus.cc:1081] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:48.764703 8044 raft_consensus.cc:1240] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:58:48.766620 8044 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:58:48.788813 8038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:58:48.789390 8038 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:48.789722 8038 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:48.792363 7961 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:48.795624 7867 catalog_manager.cc:5696] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 2 to 3, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:48.800217 10203 raft_consensus.cc:993] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:48.800812 10184 raft_consensus.cc:1081] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:48.817673 10221 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260616 19:58:48.834617 10184 raft_consensus.cc:993] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:48.853153 10184 raft_consensus.cc:1081] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:48.841917 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:48.854422 7968 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:48.854822 7968 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:48.849409 7864 catalog_manager.cc:5696] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 2 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:48.843571 7865 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:48.875308 7968 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.877254 7968 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:48.816103 8038 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.879454 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:48.880112 8111 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:48.884264 8044 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:48.890461 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:48.891247 8040 raft_consensus.cc:3055] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:48.891561 8040 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:48.893396 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:48.894090 8116 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:48.896272 8038 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:48.896430 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:48.897133 7969 raft_consensus.cc:3055] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:48.897430 7963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:48.898005 7963 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:48.898455 7963 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:48.897435 7969 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:48.900573 7969 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.902686 8040 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.904191 8040 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:48.912760 7969 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:48.917907 8116 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:48.920429 8009 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:48.921744 10202 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:48.922289 10202 raft_consensus.cc:697] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:48.923134 10202 consensus_queue.cc:237] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:48.924788 7933 consensus_peers.cc:597] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43900f33cd8249e2a494e6f9dff7b254. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:48.930121 8116 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:48.935539 8111 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:48.939352 8044 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:48.936655 7934 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:48.942262 10221 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:48.942739 10221 raft_consensus.cc:697] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:48.943266 7969 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:48.939095 7963 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.944988 10221 consensus_queue.cc:237] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.945858 8040 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:48.946832 7963 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:48.950083 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:48.951721 8111 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:48.953995 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:48.954969 8040 raft_consensus.cc:3055] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:48.955739 8040 raft_consensus.cc:740] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:48.961395 8040 consensus_queue.cc:260] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:48.967485 8040 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:48.973199 8111 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:48.974881 7934 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:48.949798 10228 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260616 19:58:48.981930 10172 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:48.990085 8040 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:49.022296 10116 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:49.031194 10172 raft_consensus.cc:697] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:49.032637 10172 consensus_queue.cc:237] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:49.127158 10228 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:49.132893 8044 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
W20260616 19:58:49.136421 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 73795us
I20260616 19:58:49.193091 7968 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260616 19:58:49.197154 10188 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:49.205124 8115 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:49.218474 8039 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:49.232775 7866 catalog_manager.cc:5696] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:49.239975 7866 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:49.245854 8115 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260616 19:58:49.249778 10195 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:49.253273 10188 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:58:49.264011 7963 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:49.276603 10195 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260616 19:58:49.329284 7969 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:49.352589 7866 catalog_manager.cc:5696] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 4, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:49.358574 7866 catalog_manager.cc:5696] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:49.415571 7969 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:49.423766 8039 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260616 19:58:49.421825 10252 log.cc:927] Time spent T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260616 19:58:49.419286 10245 ts_tablet_manager.cc:933] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:58:49.427554 8115 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260616 19:58:49.428050 7933 consensus_peers.cc:597] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a82eaaad5b1444b9ca242cd0aaeed47. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:49.421078 10202 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260616 19:58:49.458894 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 94007.5us
I20260616 19:58:49.447134 10221 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260616 19:58:49.477548 10116 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:49.479590 8115 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:49.491083 10196 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:49.601245 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:49.604389 10245 tablet_copy_client.cc:323] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
I20260616 19:58:49.629729 8043 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:49.645798 8044 raft_consensus.cc:1275] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:49.676558 10272 ts_tablet_manager.cc:933] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:58:49.662175 7980 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet 3e92f5df5ec84f069decf29684f08b06 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
I20260616 19:58:49.681128 7980 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet 3e92f5df5ec84f069decf29684f08b06 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-3e92f5df5ec84f069decf29684f08b06
I20260616 19:58:49.684751 10236 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:49.673748 10237 consensus_queue.cc:1048] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260616 19:58:49.704664 8115 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:49.668849 10172 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
W20260616 19:58:49.722380 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 57145.1us
I20260616 19:58:49.725582 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:49.726882 7980 tablet_copy_source_session.cc:215] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:49.743539 10268 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:58:49.750568 7963 raft_consensus.cc:606] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:49.755472 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 84125.2us
I20260616 19:58:49.757476 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:49.758045 8044 raft_consensus.cc:606] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:49.764854 8115 raft_consensus.cc:1275] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:49.773484 10245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e92f5df5ec84f069decf29684f08b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:49.801810 8043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:49.802707 8043 raft_consensus.cc:606] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:49.809666 10253 consensus_queue.cc:1048] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:58:49.812222 8110 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:49.814844 7968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:49.815606 7968 raft_consensus.cc:606] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:49.815933 8117 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:49.825456 10262 raft_consensus.cc:993] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:49.825946 10262 raft_consensus.cc:1081] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:49.837122 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:49.838150 8115 raft_consensus.cc:606] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:49.851364 9951 raft_consensus.cc:993] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:49.853549 9951 raft_consensus.cc:1081] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:49.853005 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:49.856374 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:49.857298 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:49.803061 10272 tablet_copy_client.cc:323] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
I20260616 19:58:49.863307 8044 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:49.864890 8043 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:58:49.884613 8043 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:49.885027 8043 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:49.893488 7980 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet 43900f33cd8249e2a494e6f9dff7b254 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
W20260616 19:58:49.893939 8082 consensus_peers.cc:597] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7c5954a3dd147cea894b8e392ab2bd6. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:49.894088 7980 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet 43900f33cd8249e2a494e6f9dff7b254 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-43900f33cd8249e2a494e6f9dff7b254
I20260616 19:58:49.894105 8039 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:49.893519 7963 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:49.895939 10153 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:49.897547 7969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:49.897991 7969 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:49.898334 7969 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:49.899279 8043 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:49.909231 7969 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:49.911082 7969 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:49.912287 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:49.912983 8110 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:49.920018 10207 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:49.926012 8110 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:49.927294 7934 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:49.929481 10274 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:49.929984 10274 raft_consensus.cc:697] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:49.931137 10274 consensus_queue.cc:237] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:49.929605 7980 tablet_copy_source_session.cc:215] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:49.932483 7864 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:49.945053 8043 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:49.947067 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:49.947811 8116 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:49.959378 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:49.960049 8038 raft_consensus.cc:3055] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:49.960368 8038 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:49.961052 8038 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:49.967067 8038 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:49.977823 8116 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:58:49.981787 8009 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:49.982132 8038 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:58:49.982692 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:49.986136 9951 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:58:49.992324 10272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43900f33cd8249e2a494e6f9dff7b254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:50.003979 7967 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:58:50.023844 7967 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:50.024775 7967 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.026479 7967 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:50.033911 10281 ts_tablet_manager.cc:933] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:58:50.035269 7967 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:58:50.041883 9951 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:50.043191 9951 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.049378 10229 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:50.050551 8044 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:50.060490 10261 raft_consensus.cc:993] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:50.061058 10261 raft_consensus.cc:1081] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:50.061780 10245 tablet_copy_client.cc:806] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:50.062960 8043 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:50.063691 8043 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:50.064566 8043 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:50.062398 10245 tablet_copy_client.cc:670] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
W20260616 19:58:50.064612 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 70112us
I20260616 19:58:50.085844 8043 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.090752 8043 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:50.091604 7969 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Committing config change with OpId 7.5: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:50.092751 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:50.096717 7968 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:50.097337 10245 tablet_copy_client.cc:538] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:50.113085 10281 tablet_copy_client.cc:323] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
I20260616 19:58:50.116825 7980 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet 3a82eaaad5b1444b9ca242cd0aaeed47 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
I20260616 19:58:50.117412 7980 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet 3a82eaaad5b1444b9ca242cd0aaeed47 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-3a82eaaad5b1444b9ca242cd0aaeed47
W20260616 19:58:50.125303 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44512us
I20260616 19:58:50.127925 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:50.128597 8110 raft_consensus.cc:3055] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:50.128944 8110 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:50.129688 8110 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.130915 8110 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:50.131244 7980 tablet_copy_source_session.cc:215] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:50.138815 10281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a82eaaad5b1444b9ca242cd0aaeed47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:50.149613 7968 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:50.155625 10245 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:50.168783 8011 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:50.181738 9951 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:50.182235 9951 raft_consensus.cc:697] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:50.182977 9951 consensus_queue.cc:237] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.214803 10202 raft_consensus.cc:993] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:50.215461 10290 raft_consensus.cc:1081] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:50.167091 10272 tablet_copy_client.cc:806] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:50.217470 10272 tablet_copy_client.cc:670] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:50.235651 10272 tablet_copy_client.cc:538] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:50.248250 8110 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:50.250108 10255 raft_consensus.cc:993] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:50.251281 10275 raft_consensus.cc:1081] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:50.256896 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:50.257519 7968 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:50.257942 7968 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:50.264721 8116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:50.265312 8116 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:50.266189 8116 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:50.286856 8116 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.294263 7866 catalog_manager.cc:5696] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index -1 to 5, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:50.289683 7968 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.321026 10281 tablet_copy_client.cc:806] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:50.326948 10281 tablet_copy_client.cc:670] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:50.328002 7968 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:50.327334 7866 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 7 to 8, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 8 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:50.329515 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:50.329269 7866 catalog_manager.cc:5696] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 4 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:50.330120 8043 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:50.331197 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:50.331455 10281 tablet_copy_client.cc:538] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:50.331772 8110 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:50.337419 8043 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:50.339028 8084 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:50.342964 10229 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:50.343439 10229 raft_consensus.cc:697] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:50.343583 8116 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:50.344338 10229 consensus_queue.cc:237] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.347856 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:50.349030 7968 raft_consensus.cc:3055] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:50.354596 7968 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:50.355475 7968 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.361027 7968 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:50.366241 7864 catalog_manager.cc:5696] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 4, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:50.362457 10272 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:50.361732 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:50.369041 8043 raft_consensus.cc:3055] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:50.369381 8043 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:50.370234 8043 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.373871 8110 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:50.375075 7934 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:50.371625 8043 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:50.358796 10281 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:50.385061 8043 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:50.384291 10276 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:50.388141 10276 raft_consensus.cc:697] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:58:50.389205 10245 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.063s user 0.002s sys 0.001s
I20260616 19:58:50.389526 10276 consensus_queue.cc:237] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:50.387416 7968 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:58:50.392776 8043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:50.393555 8043 raft_consensus.cc:606] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Received request to transfer leadership
I20260616 19:58:50.401363 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:50.402068 8044 raft_consensus.cc:606] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Received request to transfer leadership
W20260616 19:58:50.404189 7854 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 3a82eaaad5b1444b9ca242cd0aaeed47 already in progress
I20260616 19:58:50.408842 10296 ts_tablet_manager.cc:933] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:50.413638 8044 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:50.418679 7866 catalog_manager.cc:5696] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 4 to 5, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:50.425050 10228 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260616 19:58:50.454507 8112 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:58:50.455945 10202 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:50.459219 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:50.460501 8044 raft_consensus.cc:606] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Received request to transfer leadership
I20260616 19:58:50.469652 10245 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=1 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:50.470750 10245 tablet_bootstrap.cc:492] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:50.471722 10245 ts_tablet_manager.cc:1403] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.316s user 0.035s sys 0.010s
I20260616 19:58:50.474856 10245 raft_consensus.cc:359] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.475431 10245 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:50.475947 10245 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.478353 10245 ts_tablet_manager.cc:1434] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260616 19:58:50.478396 10233 raft_consensus.cc:993] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:50.479447 10233 raft_consensus.cc:1081] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:50.479874 7968 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:58:50.482286 10267 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:50.489928 10296 tablet_copy_client.cc:323] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.131:40887
I20260616 19:58:50.490238 8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53916
I20260616 19:58:50.490864 8115 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:50.491212 8115 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:50.489209 7980 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-3e92f5df5ec84f069decf29684f08b06 received from {username='slave'} at 127.0.0.1:37398
I20260616 19:58:50.499689 7980 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-3e92f5df5ec84f069decf29684f08b06 on tablet 3e92f5df5ec84f069decf29684f08b06 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:58:50.500908 7916 auto_rebalancer.cc:855] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260616 19:58:50.501336 7916 auto_rebalancer.cc:863] Could not move replica: Remote error: Service unavailable: leadership transfer for 3a82eaaad5b1444b9ca242cd0aaeed47 already in progress
W20260616 19:58:50.501612 7916 auto_rebalancer.cc:301] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 3a82eaaad5b1444b9ca242cd0aaeed47 already in progress
I20260616 19:58:50.506289 8129 tablet_copy_service.cc:140] P 6827184e3cef42c2ada15b2d31afee4b: Received BeginTabletCopySession request for tablet f7c5954a3dd147cea894b8e392ab2bd6 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:34110)
I20260616 19:58:50.506958 8129 tablet_copy_service.cc:161] P 6827184e3cef42c2ada15b2d31afee4b: Beginning new tablet copy session on tablet f7c5954a3dd147cea894b8e392ab2bd6 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:34110: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-f7c5954a3dd147cea894b8e392ab2bd6
W20260616 19:58:50.508585 8008 consensus_peers.cc:597] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:50.509300 8115 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.511400 8115 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:50.516420 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:50.516413 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:50.517009 7963 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:50.517407 8043 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:58:50.517769 8043 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:50.518658 8043 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.525257 8043 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:50.537573 8043 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:58:50.548132 7963 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:58:50.536449 7866 catalog_manager.cc:5696] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 7, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:50.549464 8085 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:50.552356 10153 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:58:50.557337 10153 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:50.544121 8129 tablet_copy_source_session.cc:215] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:58:50.592705 8110 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:50.564641 10153 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.599364 10253 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260616 19:58:50.635682 7968 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:50.636432 10296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7c5954a3dd147cea894b8e392ab2bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:58:50.637418 10203 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:50.669440 7865 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 8 to 9, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 9 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:50.683959 8819 raft_consensus.cc:1217] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Deduplicated request from leader. Original: 5.4->[5.5-5.5] Dedup: 5.5->[]
I20260616 19:58:50.780576 8110 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:50.782454 10203 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:50.813995 8115 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:50.815362 10237 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:50.842860 10281 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=1 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:50.843974 10281 tablet_bootstrap.cc:492] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:50.849705 10281 ts_tablet_manager.cc:1403] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.491s user 0.018s sys 0.022s
I20260616 19:58:50.852869 10272 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:50.857154 10296 tablet_copy_client.cc:806] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:58:50.876669 10272 tablet_bootstrap.cc:492] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:50.877701 10296 tablet_copy_client.cc:670] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:58:50.883314 10296 tablet_copy_client.cc:538] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:58:50.885670 8043 raft_consensus.cc:1275] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:50.890532 10281 raft_consensus.cc:359] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.892244 8044 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:50.899454 10281 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:50.904237 10281 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.903239 10305 consensus_queue.cc:1048] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:58:50.877650 10272 ts_tablet_manager.cc:1403] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.515s user 0.038s sys 0.017s
I20260616 19:58:50.900097 10229 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:50.917068 7968 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:58:50.918573 10272 raft_consensus.cc:359] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.924832 10272 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:50.925879 10153 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:50.926802 10272 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:50.923918 10281 ts_tablet_manager.cc:1434] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.060s user 0.007s sys 0.000s
I20260616 19:58:50.935425 7980 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-3a82eaaad5b1444b9ca242cd0aaeed47 received from {username='slave'} at 127.0.0.1:37398
I20260616 19:58:50.935878 7980 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-3a82eaaad5b1444b9ca242cd0aaeed47 on tablet 3a82eaaad5b1444b9ca242cd0aaeed47 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:50.939641 10272 ts_tablet_manager.cc:1434] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260616 19:58:50.971215 7980 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-43900f33cd8249e2a494e6f9dff7b254 received from {username='slave'} at 127.0.0.1:37398
I20260616 19:58:50.971740 7980 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-43900f33cd8249e2a494e6f9dff7b254 on tablet 43900f33cd8249e2a494e6f9dff7b254 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:50.984012 10296 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:58:51.004994 10237 raft_consensus.cc:1064] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:51.009698 8819 raft_consensus.cc:1217] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Deduplicated request from leader. Original: 7.4->[7.5-7.5] Dedup: 7.5->[]
I20260616 19:58:51.007717 10237 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:51.031199 7963 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260616 19:58:51.035120 10229 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:51.051347 8043 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260616 19:58:51.051579 8819 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Advancing to term 9
I20260616 19:58:51.061801 10207 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:51.069974 8819 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260616 19:58:51.079631 8817 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:51.085932 8115 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:51.086091 8044 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260616 19:58:51.118633 10305 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:51.119649 10237 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:51.118134 10261 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260616 19:58:51.131630 10237 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260616 19:58:51.293596 10315 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.077s user 0.000s sys 0.001s
I20260616 19:58:51.348814 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:51.349524 10253 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:51.352087 8817 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:51.364019 10207 raft_consensus.cc:1064] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:51.368752 10207 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:58:51.434217 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 58946.5us
W20260616 19:58:51.431120 10296 log.cc:927] Time spent T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.063s user 0.004s sys 0.000s
I20260616 19:58:51.455204 10296 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=1 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:58:51.443492 8116 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:51.469954 10296 tablet_bootstrap.cc:492] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:58:51.517135 7865 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:51.591755 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 108798us
I20260616 19:58:51.585381 10296 ts_tablet_manager.cc:1403] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 0.596s user 0.040s sys 0.000s
W20260616 19:58:51.611842 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 172619us
W20260616 19:58:51.646095 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 50545.1us
I20260616 19:58:51.672768 8043 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260616 19:58:51.711544 10296 raft_consensus.cc:359] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:51.712602 10296 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:58:51.713366 10296 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:51.717517 10296 ts_tablet_manager.cc:1434] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.121s user 0.006s sys 0.000s
W20260616 19:58:51.715226 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 140536us
I20260616 19:58:51.722821 10153 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:58:51.744668 10275 raft_consensus.cc:1064] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:51.765655 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:51.768510 8820 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260616 19:58:51.747362 10275 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 3, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:51.766322 8112 raft_consensus.cc:606] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:51.792773 7968 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260616 19:58:51.802824 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 57023us
I20260616 19:58:51.798565 8036 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:51.802073 10322 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
W20260616 19:58:51.878016 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 66261.3us
W20260616 19:58:51.912359 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 108323us
I20260616 19:58:51.914266 8129 tablet_copy_service.cc:342] P 6827184e3cef42c2ada15b2d31afee4b: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-f7c5954a3dd147cea894b8e392ab2bd6 received from {username='slave'} at 127.0.0.1:34110
I20260616 19:58:51.914767 8129 tablet_copy_service.cc:434] P 6827184e3cef42c2ada15b2d31afee4b: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-f7c5954a3dd147cea894b8e392ab2bd6 on tablet f7c5954a3dd147cea894b8e392ab2bd6 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:58:51.949429 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 113375us
I20260616 19:58:51.929885 10322 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260616 19:58:51.988862 8819 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:52.007401 8820 raft_consensus.cc:1217] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Deduplicated request from leader. Original: 7.4->[7.5-7.5] Dedup: 7.5->[]
W20260616 19:58:52.015615 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 76494.5us
I20260616 19:58:52.055275 8820 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:52.109359 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 66106.6us
W20260616 19:58:52.110082 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 211128us
W20260616 19:58:52.112768 7918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 135585us
I20260616 19:58:52.123893 8039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:52.125111 8039 raft_consensus.cc:606] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:52.136689 7970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:52.137773 7970 raft_consensus.cc:606] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:58:52.143101 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 196603us
I20260616 19:58:52.163168 8117 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:52.172748 10255 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:52.194561 10341 raft_consensus.cc:993] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:52.195235 10341 raft_consensus.cc:1081] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:52.205044 10352 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:52.198016 10229 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index 5 to 8, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.201313 7970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:52.222582 7970 raft_consensus.cc:606] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:52.235123 8112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53916
I20260616 19:58:52.235999 8112 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:52.236424 8112 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:52.239027 8045 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:52.286588 8043 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.303359 10349 raft_consensus.cc:993] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:52.304204 10349 raft_consensus.cc:1081] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:52.313179 8037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:52.314033 8037 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:52.314558 8037 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:52.323356 10338 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:52.328500 10351 raft_consensus.cc:993] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:52.329041 10351 raft_consensus.cc:1081] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:52.334419 7865 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 8, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.335624 10352 raft_consensus.cc:993] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:52.336220 10337 raft_consensus.cc:1081] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
W20260616 19:58:52.355312 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 184319us
I20260616 19:58:52.356442 8112 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:52.351552 8037 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.367344 8036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:58:52.368028 8036 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:52.368431 8036 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:52.382396 8037 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:52.390547 8114 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:52.390916 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:52.391191 8114 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:52.391657 8110 raft_consensus.cc:3055] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:52.391983 8114 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:52.392009 8110 raft_consensus.cc:740] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:52.405903 8110 consensus_queue.cc:260] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.407500 8110 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:52.422659 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:52.424299 8043 raft_consensus.cc:3055] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:52.424679 8043 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:52.428294 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:52.429106 7964 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:52.433987 8112 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:52.438930 8043 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:52.440495 8043 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:52.444415 8114 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:52.472671 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:58:52.473271 8819 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:52.471060 8036 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.478698 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:52.479067 8036 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:52.479310 7961 raft_consensus.cc:3055] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:52.479696 7961 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:52.480700 7961 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.482358 7961 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:52.509780 8113 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:58:52.517076 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:52.517733 8112 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:52.518235 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8890d947e59455187876dccb01c8c06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:52.518827 8115 raft_consensus.cc:606] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:52.527491 8110 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:52.529613 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:52.530272 7968 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:58:52.530606 7968 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:52.531375 7968 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:52.533008 7961 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:58:52.533360 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:52.465473 7963 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.537580 10228 consensus_queue.cc:1059] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:58:52.545747 10229 raft_consensus.cc:993] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:52.546339 10229 raft_consensus.cc:1081] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:52.546921 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:52.549316 8044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8890d947e59455187876dccb01c8c06"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:52.550284 8044 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:52.550663 8044 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:52.564836 7968 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:52.569597 8819 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:52.569825 8043 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:52.548069 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:52.576879 7964 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:52.578300 8044 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.589442 8084 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:52.589825 10345 raft_consensus.cc:1064] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:58:52.590847 10153 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:52.591352 10153 raft_consensus.cc:697] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:52.591679 8114 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:52.592100 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:52.592739 8036 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:52.596542 8044 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:52.593894 8009 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:52.586268 8011 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:52.593168 10153 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:52.606285 8112 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
W20260616 19:58:52.589238 7855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 163150us
I20260616 19:58:52.615568 10343 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:52.616690 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:52.617311 8114 raft_consensus.cc:3055] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:52.617596 8114 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:52.627367 10341 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:52.631361 10343 raft_consensus.cc:697] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:52.632503 10343 consensus_queue.cc:237] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.634904 10341 raft_consensus.cc:697] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:52.633018 8114 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.636078 10341 consensus_queue.cc:237] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.639066 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:52.639511 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:58:52.639694 7964 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:52.640188 8820 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:58:52.647585 8036 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:52.651042 7964 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:58:52.653831 8011 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:52.628832 10228 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.659973 8110 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.661331 10345 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 3, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:52.653546 8036 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.679781 7968 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:52.685151 8114 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:58:52.704205 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 50424.4us
I20260616 19:58:52.706816 8820 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:58:52.710328 8085 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:52.711627 10202 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:52.720597 10202 raft_consensus.cc:697] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:58:52.746510 7855 catalog_manager.cc:5226] ChangeConfig:REMOVE_PEER RPC for tablet 3e92f5df5ec84f069decf29684f08b06 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b022cac8cb5e46b7a39a748e81e257a7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.747328 10329 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:58:52.747928 10329 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:52.748844 10329 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:52.750818 8820 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.773725 10202 consensus_queue.cc:237] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:52.775969 8114 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
W20260616 19:58:52.804615 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 69611.6us
I20260616 19:58:52.821290 8820 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:52.824699 10331 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:52.829902 8036 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:52.831547 7963 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:52.833801 10258 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:58:52.836570 10229 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:52.868501 7865 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:52.881078 10365 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.102s user 0.001s sys 0.000s
I20260616 19:58:52.945425 8110 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:58:52.958514 10055 log.cc:927] Time spent T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Append to log took a long time: real 0.061s user 0.002s sys 0.007s
I20260616 19:58:52.963981 8819 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260616 19:58:52.965718 10345 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260616 19:58:52.977959 10345 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.979413 7864 catalog_manager.cc:5696] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 6 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 7 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260616 19:58:52.982483 10370 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.081s user 0.002s sys 0.000s
I20260616 19:58:52.983223 8819 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:52.993171 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 3a82eaaad5b1444b9ca242cd0aaeed47 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:52.994202 7964 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260616 19:58:52.995282 10371 log.cc:927] Time spent T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.079s user 0.003s sys 0.000s
I20260616 19:58:52.981288 7864 catalog_manager.cc:5696] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 3 to 4, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:53.000793 10329 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:53.002591 7864 catalog_manager.cc:5696] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 3 to 4, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 4 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:53.009035 8820 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:53.022738 10229 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260616 19:58:53.029956 7964 raft_consensus.cc:2955] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:53.053500 7968 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:53.061050 8820 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:53.061198 7866 catalog_manager.cc:5696] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 5 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:53.063313 7866 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 5 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:53.064023 10345 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:53.076484 8036 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:53.087669 10345 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260616 19:58:53.117090 7865 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 8 to 9, VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) evicted. New cstate: current_term: 9 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:53.120815 8039 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:53.128660 7964 raft_consensus.cc:1275] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:58:53.131527 10356 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:53.194263 8114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:53.195182 8114 raft_consensus.cc:606] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Received request to transfer leadership
I20260616 19:58:53.196015 7968 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:53.201450 7961 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:53.216857 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:53.217465 8110 raft_consensus.cc:606] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Received request to transfer leadership
I20260616 19:58:53.234118 8114 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:53.223995 10331 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260616 19:58:53.259919 8110 raft_consensus.cc:1275] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260616 19:58:53.260218 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 51095.4us
I20260616 19:58:53.265290 8819 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:53.274542 7864 catalog_manager.cc:5696] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index 5 to 6, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:53.294034 7967 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:58:53.308269 10332 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260616 19:58:53.301052 10330 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260616 19:58:53.318320 7853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 95222.9us
I20260616 19:58:53.290339 10341 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260616 19:58:53.354604 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:53.355392 8115 raft_consensus.cc:606] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Received request to transfer leadership
W20260616 19:58:53.360853 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 64408.9us
W20260616 19:58:53.421417 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 56826.8us
W20260616 19:58:53.499871 10055 log.cc:927] Time spent T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
W20260616 19:58:53.544071 10374 log.cc:927] Time spent T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.101s user 0.001s sys 0.000s
W20260616 19:58:53.478430 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 90669.2us
I20260616 19:58:53.569099 8028 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a82eaaad5b1444b9ca242cd0aaeed47 with delete_type TABLET_DATA_TOMBSTONED (TS de4c5ce3ea8345ae9068f825b8d60e19 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51868
I20260616 19:58:53.601663 10325 raft_consensus.cc:993] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:53.609213 10398 tablet_replica.cc:333] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: stopping tablet replica
I20260616 19:58:53.565092 10356 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260616 19:58:53.623791 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 201244us
I20260616 19:58:53.624011 8044 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260616 19:58:53.660640 10399 raft_consensus.cc:1081] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
W20260616 19:58:53.654824 7853 catalog_manager.cc:4754] TS 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): ChangeConfig:REMOVE_PEER RPC failed for tablet 3e92f5df5ec84f069decf29684f08b06: Remote error: Service unavailable: leader transfer in progress
I20260616 19:58:53.722611 10398 raft_consensus.cc:2243] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:53.747224 7967 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:53.747828 7967 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:53.748201 7967 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:53.774842 7967 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:53.782240 8039 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:58:53.787199 8818 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260616 19:58:53.796377 10398 raft_consensus.cc:2272] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:53.832190 10389 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260616 19:58:53.853710 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 44730.5us
I20260616 19:58:53.860236 10351 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260616 19:58:53.892838 10401 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:58:53.897496 7967 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:58:53.898150 7916 auto_rebalancer.cc:855] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260616 19:58:53.899257 7916 auto_rebalancer.cc:863] Could not move replica: Remote error: Service unavailable: leadership transfer for 3e92f5df5ec84f069decf29684f08b06 already in progress
W20260616 19:58:53.899564 7916 auto_rebalancer.cc:301] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 3e92f5df5ec84f069decf29684f08b06 already in progress
I20260616 19:58:53.900722 8110 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:53.908607 10398 ts_tablet_manager.cc:1916] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:53.919178 8115 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:53.926326 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:58:53.926982 8818 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:53.935850 8818 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:53.952194 8039 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260616 19:58:53.952675 7964 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:58:53.964651 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:53.965734 10258 consensus_queue.cc:1059] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:53.990916 7961 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:53.997700 8819 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260616 19:58:54.000660 8115 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:54.001215 8115 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:54.004771 8115 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:54.010084 10388 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Committing config change with OpId 7.7: config changed from index 6 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:54.012257 10391 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260616 19:58:54.006153 8115 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:54.047319 8818 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 6 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:54.073282 8115 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:54.071558 7864 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 6 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:54.075309 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 62443.7us
W20260616 19:58:54.081575 10399 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260616 19:58:54.086692 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 90459.1us
W20260616 19:58:54.127913 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 147216us
W20260616 19:58:54.155951 7853 catalog_manager.cc:5226] ChangeConfig:REMOVE_PEER RPC for tablet 3e92f5df5ec84f069decf29684f08b06 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6827184e3cef42c2ada15b2d31afee4b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:54.200776 7853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 57955.5us
W20260616 19:58:54.224452 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 119277us
W20260616 19:58:54.224676 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 80714.4us
I20260616 19:58:54.231271 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:54.231982 8036 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:54.148276 10398 ts_tablet_manager.cc:1929] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 9.8
I20260616 19:58:54.273448 10398 log.cc:1199] T 3a82eaaad5b1444b9ca242cd0aaeed47 P de4c5ce3ea8345ae9068f825b8d60e19: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-1-root/wals/3a82eaaad5b1444b9ca242cd0aaeed47
I20260616 19:58:54.274178 10406 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:54.274984 10406 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:54.276579 7854 catalog_manager.cc:5027] TS de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): tablet 3a82eaaad5b1444b9ca242cd0aaeed47 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:54.281852 10387 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
W20260616 19:58:54.283694 10394 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.070s user 0.001s sys 0.000s
I20260616 19:58:54.179569 10325 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260616 19:58:54.298847 10345 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260616 19:58:54.261387 10392 log.cc:927] Time spent T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260616 19:58:54.321835 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:54.367067 8036 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:58:54.395678 10413 log.cc:927] Time spent T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.123s user 0.000s sys 0.003s
W20260616 19:58:54.397667 10408 log.cc:927] Time spent T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.124s user 0.001s sys 0.000s
W20260616 19:58:54.437119 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 49604.7us
I20260616 19:58:54.504688 10406 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:54.513083 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:54.517378 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:54.521296 8041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:54.522240 8041 raft_consensus.cc:606] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:54.532244 10419 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:54.533191 10419 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:54.534297 10419 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:58:54.546762 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 270338us
W20260616 19:58:54.548070 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 322413us
W20260616 19:58:54.555009 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 289177us
I20260616 19:58:54.577139 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:54.592896 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:54.648072 8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:54.648736 8117 raft_consensus.cc:606] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Received request to transfer leadership to TS c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:58:54.672670 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 75576us
W20260616 19:58:54.682212 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 210901us
W20260616 19:58:54.697950 10420 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.050s user 0.001s sys 0.000s
I20260616 19:58:54.716904 7964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:54.717593 7964 raft_consensus.cc:606] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:54.710108 8082 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:54.734019 10331 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:58:54.735841 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 116341us
I20260616 19:58:54.748083 7964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:54.748796 7964 raft_consensus.cc:606] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:58:54.768541 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 49350.5us
I20260616 19:58:54.706949 10388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49182 (request call id 73) took 1590 ms. Trace:
W20260616 19:58:54.795393 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 52394.2us
W20260616 19:58:54.786279 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 122239us
W20260616 19:58:54.817294 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 195628us
W20260616 19:58:54.815326 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45378us
W20260616 19:58:54.863451 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 54697.2us
W20260616 19:58:54.865990 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 66994.4us
I20260616 19:58:54.867347 8113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:54.868204 8113 raft_consensus.cc:606] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:54.874332 10258 raft_consensus.cc:993] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:54.874882 10417 raft_consensus.cc:1081] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Signalling peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 to start an election
I20260616 19:58:54.822229 10337 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:54.876513 10337 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:54.877218 10320 raft_consensus.cc:993] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:54.877707 10320 raft_consensus.cc:1081] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:54.878641 10337 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:54.888000 8817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47"
dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
from {username='slave'} at 127.0.0.1:52458
I20260616 19:58:54.888710 8817 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:54.889089 8817 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:58:54.888461 8038 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 6 to 7, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:54.933945 10203 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:54.896400 10345 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Committing config change with OpId 7.7: config changed from index 6 to 7, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:54.934763 10203 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:54.938191 10203 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:54.784613 10388 rpcz_store.cc:276] 0616 19:58:53.110622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:53.124701 (+ 14079us) service_pool.cc:224] Handling call
0616 19:58:53.300066 (+175365us) log.cc:844] Serialized 501 byte log entry
0616 19:58:53.581336 (+281270us) spinlock_profiling.cc:243] Waited 916 us on lock 0x7b18003cd7e0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:54.700473 (+1119137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1074,"raft.queue_time_us":1390,"raft.run_cpu_time_us":1269,"raft.run_wall_time_us":7130,"spinlock_wait_cycles":2126336}
I20260616 19:58:54.954401 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:54.939759 8817 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:54.978549 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:54.981377 8817 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:54.976941 7855 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 43900f33cd8249e2a494e6f9dff7b254 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:58:54.987592 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:54.994091 8117 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:58:54.994493 8117 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:54.997637 8817 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 9. Current term is 10. Ops: []
I20260616 19:58:54.995473 8117 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:55.008270 8117 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:58:55.024029 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:55.024437 8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:55.025033 8115 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:55.025424 8115 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:55.038760 10345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53906 (request call id 76) took 1119 ms. Trace:
I20260616 19:58:55.039194 10345 rpcz_store.cc:276] 0616 19:58:53.916607 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:53.916818 (+ 211us) service_pool.cc:224] Handling call
0616 19:58:53.922333 (+ 5515us) log.cc:844] Serialized 501 byte log entry
0616 19:58:55.027221 (+1104888us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1532,"raft.run_cpu_time_us":1039,"raft.run_wall_time_us":1038,"spinlock_wait_cycles":5888}
W20260616 19:58:55.038579 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 58255us
I20260616 19:58:55.043479 8008 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:58:55.047678 8817 raft_consensus.cc:2955] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 6 to 7, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:55.045866 7853 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet f7c5954a3dd147cea894b8e392ab2bd6 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260616 19:58:55.054757 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 61619.3us
I20260616 19:58:55.084034 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:55.098500 8038 raft_consensus.cc:2955] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 6 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:55.105115 8115 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.110247 8115 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:55.113741 10432 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260616 19:58:55.115203 10443 log.cc:927] Time spent T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.110s user 0.000s sys 0.001s
I20260616 19:58:55.122282 10438 raft_consensus.cc:993] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:55.122808 10438 raft_consensus.cc:1081] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:55.129458 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:55.130101 7968 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:58:55.130434 7968 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:55.131124 7968 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.132179 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 84) took 1062 ms. Trace:
I20260616 19:58:55.133740 7867 catalog_manager.cc:5696] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index 6 to 7, VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) evicted. New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:55.136252 8101 tablet_service.cc:1558] Processing DeleteTablet for tablet 43900f33cd8249e2a494e6f9dff7b254 with delete_type TABLET_DATA_TOMBSTONED (TS 6827184e3cef42c2ada15b2d31afee4b not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:55.136891 8817 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:58:55.136718 7865 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 7, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:55.131971 7968 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:55.152508 7961 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:55.153149 7961 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:55.153481 7961 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:55.181833 10428 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260616 19:58:55.198041 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:58:55.206003 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 64632us
I20260616 19:58:55.209450 8117 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
I20260616 19:58:55.220444 7968 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
W20260616 19:58:55.228538 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 147672us
I20260616 19:58:55.132562 7864 rpcz_store.cc:276] 0616 19:58:54.069414 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:54.069569 (+ 155us) service_pool.cc:224] Handling call
0616 19:58:55.132079 (+1062510us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:54.356746 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:54.357483 (+ 737us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:54.357534 (+ 51us) write_op.cc:435] Acquired schema lock
0616 19:58:54.357560 (+ 26us) tablet.cc:662] Decoding operations
0616 19:58:54.357987 (+ 427us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:54.358082 (+ 95us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:54.358131 (+ 49us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:54.358367 (+ 236us) tablet.cc:701] Row locks acquired
0616 19:58:54.358391 (+ 24us) write_op.cc:260] PREPARE: finished
0616 19:58:54.358571 (+ 180us) write_op.cc:270] Start()
0616 19:58:54.358708 (+ 137us) write_op.cc:276] Timestamp: P: 1781639934358513 usec, L: 0
0616 19:58:54.358738 (+ 30us) op_driver.cc:348] REPLICATION: starting
0616 19:58:54.359539 (+ 801us) log.cc:844] Serialized 536 byte log entry
0616 19:58:54.676340 (+316801us) op_driver.cc:464] REPLICATION: finished
0616 19:58:54.969808 (+293468us) write_op.cc:301] APPLY: starting
0616 19:58:54.969921 (+ 113us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:54.970052 (+ 131us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:54.970214 (+ 162us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:54.970463 (+ 249us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:54.970674 (+ 211us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:54.970753 (+ 79us) write_op.cc:312] APPLY: finished
0616 19:58:54.992274 (+ 21521us) log.cc:844] Serialized 23 byte log entry
0616 19:58:55.011207 (+ 18933us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:55.017149 (+ 5942us) write_op.cc:454] Released schema lock
0616 19:58:55.020854 (+ 3705us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:55.080642 (+ 59788us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:55.086027 (+ 5385us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15750,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":288769,"apply.run_cpu_time_us":2206,"apply.run_wall_time_us":86197,"num_ops":1,"prepare.queue_time_us":270757,"prepare.run_cpu_time_us":3128,"prepare.run_wall_time_us":3128,"replication_time_us":289966,"thread_start_us":276551,"threads_started":2}]]}
I20260616 19:58:55.239248 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:55.239868 7964 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:58:55.239601 7961 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.246431 7961 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:55.248787 8085 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:55.244961 8784 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:58:55.253899 10445 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:58:55.262638 8040 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:58:55.268730 7935 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:55.278952 7964 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
I20260616 19:58:55.272218 10441 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:55.284912 10441 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:55.290885 8119 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:58:55.293977 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:55.294520 8115 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:55.274160 10330 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:58:55.274672 10352 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:58:55.286403 10445 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:58:55.305519 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:58:55.310775 10330 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:55.311347 10393 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:58:55.311700 10330 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.316437 10445 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:55.309263 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:55.324630 8040 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:58:55.337343 10321 raft_consensus.cc:993] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:55.338310 10321 raft_consensus.cc:1081] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:55.348820 8115 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:55.361145 8113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:55.361799 8113 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:55.362224 8113 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:55.364813 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:58:55.365061 10419 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:58:55.365577 10441 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:58:55.366511 10448 log.cc:927] Time spent T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.059s user 0.000s sys 0.001s
I20260616 19:58:55.361668 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:55.367611 8110 raft_consensus.cc:606] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:55.375779 10450 tablet_replica.cc:333] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: stopping tablet replica
I20260616 19:58:55.376554 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:55.376993 10450 raft_consensus.cc:2243] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:55.377301 8038 raft_consensus.cc:3055] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:58:55.377925 8038 raft_consensus.cc:740] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:55.378178 10450 raft_consensus.cc:2272] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:55.378854 8038 consensus_queue.cc:260] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.380213 8038 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:58:55.394266 8040 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:58:55.395009 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260616 19:58:55.403403 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:55.399720 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:55.404579 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:55.401273 8038 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:58:55.403036 7934 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:58:55.384470 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 44482.9us
I20260616 19:58:55.388216 8113 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.381675 10450 ts_tablet_manager.cc:1916] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:55.421136 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:55.422041 7967 raft_consensus.cc:3055] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:58:55.421401 8113 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:55.422402 7967 raft_consensus.cc:740] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:55.418444 7935 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:55.435542 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:55.436304 8036 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:55.423406 7967 consensus_queue.cc:260] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.438619 7967 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:58:55.446146 10320 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:58:55.470116 10253 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:58:55.474920 8036 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:55.482641 8084 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:55.492731 10229 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:58:55.493166 10229 raft_consensus.cc:697] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:55.499555 7967 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:58:55.498518 10229 consensus_queue.cc:237] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.543910 7958 tablet_service.cc:1558] Processing DeleteTablet for tablet f7c5954a3dd147cea894b8e392ab2bd6 with delete_type TABLET_DATA_TOMBSTONED (TS b022cac8cb5e46b7a39a748e81e257a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:49182
I20260616 19:58:55.582952 10461 tablet_replica.cc:333] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: stopping tablet replica
I20260616 19:58:55.610903 7864 catalog_manager.cc:5696] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 9 to 10, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 10 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:58:55.598589 10320 raft_consensus.cc:697] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:55.657299 10320 consensus_queue.cc:237] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.666071 10450 ts_tablet_manager.cc:1929] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.6
I20260616 19:58:55.666558 10450 log.cc:1199] T 43900f33cd8249e2a494e6f9dff7b254 P 6827184e3cef42c2ada15b2d31afee4b: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root/wals/43900f33cd8249e2a494e6f9dff7b254
I20260616 19:58:55.679473 7853 catalog_manager.cc:5027] TS 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): tablet 43900f33cd8249e2a494e6f9dff7b254 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:55.681851 10325 raft_consensus.cc:993] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:55.682359 10325 raft_consensus.cc:1081] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:55.684751 7964 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:55.685299 7964 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:55.685556 10401 raft_consensus.cc:993] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:55.685667 7964 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:55.686100 10401 raft_consensus.cc:1081] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:55.690135 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:55.693081 7968 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:55.693547 7968 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:55.716849 7865 catalog_manager.cc:5696] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 8 to 9, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 9 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:55.721980 7968 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.732641 7968 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:55.719448 7865 catalog_manager.cc:5696] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 3 to 4, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 4 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:55.747259 7964 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.748682 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:55.749552 8111 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:58:55.749872 8111 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:55.783144 8111 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.838351 7964 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:55.856642 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:55.857489 8113 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:55.857858 8113 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:55.860633 8111 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:55.869077 7968 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:55.868439 8113 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:55.899657 7964 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260616 19:58:55.998271 8113 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:56.059067 7968 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260616 19:58:56.069689 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:56.069482 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:56.078711 8041 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:56.098013 8043 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260616 19:58:56.098974 8036 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:56.101125 8043 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260616 19:58:56.104449 10461 raft_consensus.cc:2243] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Raft consensus shutting down.
W20260616 19:58:56.108657 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 93140.6us
I20260616 19:58:56.114661 10461 raft_consensus.cc:2272] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Raft consensus is shut down!
W20260616 19:58:56.115604 10465 log.cc:927] Time spent T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260616 19:58:56.124607 8113 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:56.136518 10445 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260616 19:58:56.138970 8819 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260616 19:58:56.143787 8041 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
W20260616 19:58:56.158085 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:56.166345 8111 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:58:56.167279 8036 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:56.168061 8040 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:58:56.181350 7964 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:56.189393 10330 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260616 19:58:56.215993 10461 ts_tablet_manager.cc:1916] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:56.192670 10330 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
W20260616 19:58:56.229517 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 129521us
I20260616 19:58:56.233784 8113 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260616 19:58:56.241479 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:56.292054 8111 raft_consensus.cc:1275] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:56.271831 7866 catalog_manager.cc:5696] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:56.301245 10471 consensus_queue.cc:1048] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260616 19:58:56.304876 7934 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:56.270581 10407 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260616 19:58:56.312601 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:56.313035 10321 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:56.269742 10427 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260616 19:58:56.340382 10468 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:58:56.340986 10468 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:56.341868 10468 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:56.313608 10321 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:56.344430 10321 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:56.345377 8036 raft_consensus.cc:1275] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:58:56.355772 8039 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:58:56.370867 10409 consensus_queue.cc:1048] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260616 19:58:56.380671 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 130162us
I20260616 19:58:56.375783 10389 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260616 19:58:56.423938 10320 consensus_queue.cc:1048] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260616 19:58:56.493013 10407 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260616 19:58:56.510737 8819 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Committing config change with OpId 7.9: config changed from index 6 to 9, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:56.508658 10337 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Committing config change with OpId 7.9: config changed from index 6 to 9, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:56.571681 10478 log.cc:927] Time spent T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.077s user 0.000s sys 0.000s
W20260616 19:58:56.584930 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 58163.7us
W20260616 19:58:56.647136 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 45339.1us
W20260616 19:58:56.661778 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 123257us
W20260616 19:58:56.732700 10481 log.cc:927] Time spent T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.075s user 0.000s sys 0.000s
I20260616 19:58:56.835474 10461 ts_tablet_manager.cc:1929] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.6
I20260616 19:58:56.839231 10461 log.cc:1199] T f7c5954a3dd147cea894b8e392ab2bd6 P b022cac8cb5e46b7a39a748e81e257a7: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-0-root/wals/f7c5954a3dd147cea894b8e392ab2bd6
I20260616 19:58:56.840556 10461 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49182 (request call id 74) took 1297 ms. Trace:
I20260616 19:58:56.840972 10461 rpcz_store.cc:276] 0616 19:58:55.543430 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:55.543644 (+ 214us) service_pool.cc:224] Handling call
0616 19:58:55.582482 (+ 38838us) ts_tablet_manager.cc:1151] Deleting tablet f7c5954a3dd147cea894b8e392ab2bd6
0616 19:58:55.582570 (+ 88us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0616 19:58:56.503431 (+920861us) tablet_metadata.cc:665] Metadata flushed
0616 19:58:56.825001 (+321570us) tablet_metadata.cc:665] Metadata flushed
0616 19:58:56.840473 (+ 15472us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":323049,"tablet-delete.queue_time_us":37943,"thread_start_us":37484,"threads_started":1}
W20260616 19:58:56.842990 10476 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.343s user 0.001s sys 0.000s
I20260616 19:58:56.857293 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260616 19:58:56.860320 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 62195.5us
W20260616 19:58:56.864274 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 64150.4us
I20260616 19:58:56.867843 10337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49182 (request call id 75) took 1079 ms. Trace:
I20260616 19:58:56.869692 10337 rpcz_store.cc:276] 0616 19:58:55.788018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:55.793963 (+ 5945us) service_pool.cc:224] Handling call
0616 19:58:55.880138 (+ 86175us) log.cc:844] Serialized 501 byte log entry
0616 19:58:55.940217 (+ 60079us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b18003683e0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:58:56.862377 (+922160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15238,"raft.queue_time_us":32069,"raft.run_cpu_time_us":15209,"raft.run_wall_time_us":246260,"spinlock_wait_cycles":6995200}
W20260616 19:58:56.876310 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 44079.6us
W20260616 19:58:56.878321 10486 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.178s user 0.000s sys 0.002s
I20260616 19:58:56.921979 8040 raft_consensus.cc:2955] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Committing config change with OpId 7.9: config changed from index 6 to 9, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:58:56.923022 10483 log.cc:927] Time spent T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.422s user 0.002s sys 0.000s
I20260616 19:58:56.925700 8036 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:56.927565 7867 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 6 to 9, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:58:56.978358 10389 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
W20260616 19:58:57.032698 10458 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
W20260616 19:58:57.056242 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 67953.7us
I20260616 19:58:57.083675 7855 catalog_manager.cc:5027] TS b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): tablet f7c5954a3dd147cea894b8e392ab2bd6 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:58:57.093847 7855 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet 3e92f5df5ec84f069decf29684f08b06 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 8)
I20260616 19:58:57.127861 7964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:57.128691 7964 raft_consensus.cc:606] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:58:57.174152 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:57.174883 8110 raft_consensus.cc:606] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:57.183743 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43378ff312f94246811895976a8715d6"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:57.184532 8110 raft_consensus.cc:606] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:57.190661 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:58:57.191332 8044 raft_consensus.cc:606] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:57.196439 10466 raft_consensus.cc:993] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:57.196985 10466 raft_consensus.cc:1081] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:57.202373 8117 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:57.226361 8119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:58:57.227025 8119 raft_consensus.cc:606] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:58:57.229068 10467 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:57.240250 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:57.240888 7968 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:57.241140 7968 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:57.232995 10467 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:57.246105 10468 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260616 19:58:57.275354 10467 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:57.280122 7968 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.312631 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:57.345402 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 64926.1us
I20260616 19:58:57.360229 7968 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:57.397200 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:57.398666 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:57.406010 8115 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:58:57.406355 8115 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:57.407105 8115 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.405262 10434 raft_consensus.cc:993] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:57.408743 10434 raft_consensus.cc:1081] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:57.407989 8115 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:57.417662 8115 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
W20260616 19:58:57.421407 10055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s user 0.022s sys 0.043s
W20260616 19:58:57.421842 10055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s user 0.022s sys 0.043s
I20260616 19:58:57.448724 8101 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e92f5df5ec84f069decf29684f08b06 with delete_type TABLET_DATA_TOMBSTONED (TS 6827184e3cef42c2ada15b2d31afee4b not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:53906
I20260616 19:58:57.462518 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 188) took 1388 ms. Trace:
I20260616 19:58:57.404474 7963 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:57.477332 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:58:57.478597 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:58:57.480831 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:58:57.482789 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:58:57.482962 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:57.483624 8038 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:58:57.480863 10389 raft_consensus.cc:993] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:57.484386 10389 raft_consensus.cc:1081] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:58:57.485364 8082 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:58:57.488320 10229 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:58:57.483652 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:58:57.505731 7934 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:57.510275 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:58:57.525605 7968 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:57.526429 7968 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:57.509328 8038 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:58:57.519327 7865 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 7 to 8, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:57.537637 7968 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.537693 8119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:58:57.542411 8119 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:57.543679 8119 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:57.551277 7968 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:57.553747 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:57.554349 8115 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:57.537683 10503 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:58:57.572235 8115 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:57.573977 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:58:57.463037 7866 rpcz_store.cc:276] 0616 19:58:56.074485 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:56.102535 (+ 28050us) service_pool.cc:224] Handling call
0616 19:58:57.462457 (+1359922us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:58:56.568886 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:58:56.702428 (+133542us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:58:56.708559 (+ 6131us) write_op.cc:435] Acquired schema lock
0616 19:58:56.714859 (+ 6300us) tablet.cc:662] Decoding operations
0616 19:58:56.728324 (+ 13465us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:58:56.728418 (+ 94us) write_op.cc:641] Partition lock acquired for write op
0616 19:58:56.728459 (+ 41us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:58:56.728681 (+ 222us) tablet.cc:701] Row locks acquired
0616 19:58:56.728711 (+ 30us) write_op.cc:260] PREPARE: finished
0616 19:58:56.728880 (+ 169us) write_op.cc:270] Start()
0616 19:58:56.729015 (+ 135us) write_op.cc:276] Timestamp: P: 1781639936728830 usec, L: 0
0616 19:58:56.729044 (+ 29us) op_driver.cc:348] REPLICATION: starting
0616 19:58:56.729845 (+ 801us) log.cc:844] Serialized 541 byte log entry
0616 19:58:57.196968 (+467123us) op_driver.cc:464] REPLICATION: finished
0616 19:58:57.304616 (+107648us) write_op.cc:301] APPLY: starting
0616 19:58:57.304728 (+ 112us) tablet.cc:1367] starting BulkCheckPresence
0616 19:58:57.304832 (+ 104us) tablet.cc:1370] finished BulkCheckPresence
0616 19:58:57.304956 (+ 124us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:58:57.305318 (+ 362us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:58:57.305486 (+ 168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:58:57.305537 (+ 51us) write_op.cc:312] APPLY: finished
0616 19:58:57.305830 (+ 293us) log.cc:844] Serialized 23 byte log entry
0616 19:58:57.306054 (+ 224us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:57.306127 (+ 73us) write_op.cc:454] Released schema lock
0616 19:58:57.306166 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0616 19:58:57.306522 (+ 356us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:58:57.306556 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":55488,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":107404,"apply.run_cpu_time_us":1664,"apply.run_wall_time_us":1902,"num_ops":1,"prepare.queue_time_us":232994,"prepare.run_cpu_time_us":3724,"prepare.run_wall_time_us":175174,"replication_time_us":467758,"thread_start_us":216229,"threads_started":2}]]}
I20260616 19:58:57.580701 10517 tablet_replica.cc:333] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: stopping tablet replica
I20260616 19:58:57.581883 10517 raft_consensus.cc:2243] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Raft consensus shutting down.
I20260616 19:58:57.582757 10517 raft_consensus.cc:2272] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Raft consensus is shut down!
I20260616 19:58:57.587489 10419 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:57.588099 10419 raft_consensus.cc:697] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:57.589011 10419 consensus_queue.cc:237] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.571482 10503 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:57.603152 10503 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.619865 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:57.620504 8039 raft_consensus.cc:3055] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:57.620805 8039 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:57.622978 7865 catalog_manager.cc:5696] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 6 to 7, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:58:57.627441 10499 raft_consensus.cc:993] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:57.627962 10499 raft_consensus.cc:1081] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:58:57.638186 10517 ts_tablet_manager.cc:1916] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:58:57.622727 8119 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.648947 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:58:57.649521 7968 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:57.649902 7968 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:57.648043 8039 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.671887 8119 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:57.674868 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:58:57.675493 7961 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:58:57.675791 7961 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:57.676810 8039 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:57.677120 7961 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.678680 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:57.684540 8036 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:57.686264 7961 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:58:57.695199 8039 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:58:57.703565 7968 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:57.702355 7933 consensus_peers.cc:597] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69c80791d0c44cb0b616bf402ae91b46. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:57.716288 10407 raft_consensus.cc:993] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:57.716807 10407 raft_consensus.cc:1081] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:58:57.716894 8036 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:57.726467 8045 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43378ff312f94246811895976a8715d6"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:58:57.727121 8045 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:58:57.727494 8045 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:57.728374 7961 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:58:57.729089 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:58:57.724792 8084 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:57.729789 8039 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:57.730546 10499 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:58:57.731096 8036 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260616 19:58:57.733731 10499 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:57.738639 7968 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:57.743274 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:57.743794 8110 raft_consensus.cc:3055] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:58:57.743993 8110 raft_consensus.cc:740] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:57.746611 8039 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:57.750931 8110 consensus_queue.cc:260] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.734668 10499 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.760391 8110 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:58:57.770135 7935 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:57.782425 8045 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:57.786136 8045 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:58:57.787761 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:58:57.788358 8819 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:57.788448 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:58:57.789064 8119 raft_consensus.cc:3055] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:58:57.789356 8119 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:58:57.795621 7867 catalog_manager.cc:5696] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 7 to 8, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:57.805663 8819 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:57.807791 8008 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:58:57.810215 10503 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260616 19:58:57.810951 10503 raft_consensus.cc:697] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:58:57.814183 8115 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260616 19:58:57.816325 8119 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:57.768093 10419 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260616 19:58:57.817516 8119 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:58:57.822750 10468 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:57.840283 8110 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:58:57.845031 8119 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:58:57.844594 10510 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:58:57.842530 10503 consensus_queue.cc:237] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:57.891892 10510 raft_consensus.cc:697] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:58:57.899610 10510 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:57.892261 7867 catalog_manager.cc:5696] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 6 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:58:57.925745 10517 ts_tablet_manager.cc:1929] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.8
I20260616 19:58:57.926342 10517 log.cc:1199] T 3e92f5df5ec84f069decf29684f08b06 P 6827184e3cef42c2ada15b2d31afee4b: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root/wals/3e92f5df5ec84f069decf29684f08b06
I20260616 19:58:57.929350 7853 catalog_manager.cc:5027] TS 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): tablet 3e92f5df5ec84f069decf29684f08b06 (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
W20260616 19:58:57.936791 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 95133us
W20260616 19:58:57.945056 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 74001.2us
I20260616 19:58:57.969128 7867 catalog_manager.cc:5696] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 7, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:57.971868 7867 catalog_manager.cc:5696] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 2, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:57.972123 7865 catalog_manager.cc:5696] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 4 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:57.989434 10468 raft_consensus.cc:2955] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Committing config change with OpId 9.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:57.990967 8115 raft_consensus.cc:2955] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Committing config change with OpId 9.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:57.998617 8040 raft_consensus.cc:2955] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Committing config change with OpId 9.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.022643 7963 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:58.037241 7865 catalog_manager.cc:5696] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 9 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.040351 8045 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:58:58.041963 10503 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260616 19:58:58.059005 7933 consensus_peers.cc:597] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f24cd55ee50d4bcfaee34b2adaa9b08a. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:58.059592 8115 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260616 19:58:58.061794 10419 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:58:58.072383 8045 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260616 19:58:58.087472 10337 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260616 19:58:58.101565 8110 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:58:58.106878 10503 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:58.172139 10419 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.180588 8110 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.186918 8040 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.189064 8041 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:58.195914 10503 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:58.204629 7963 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:58.205951 7961 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:58.208763 10406 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260616 19:58:58.222657 7933 consensus_peers.cc:597] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b14a29bdfa9541db9d3f1e93a225a342. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:58:58.226177 8111 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:58:58.226989 8045 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:58:58.228534 10427 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:58:58.229040 8115 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:58:58.229584 10468 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:58:58.230762 10419 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:58:58.232788 7865 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index -1 to 3, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 2 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:58.249203 10337 raft_consensus.cc:2955] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Committing config change with OpId 8.8: config changed from index -1 to 8, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.264930 8041 raft_consensus.cc:2955] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Committing config change with OpId 8.8: config changed from index -1 to 8, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.286145 8045 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:58:58.288570 10499 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:58:58.286201 8119 raft_consensus.cc:2955] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Committing config change with OpId 8.8: config changed from index -1 to 8, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:58:58.289003 8040 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:58.530277 8111 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260616 19:58:58.555408 7865 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index -1 to 8, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:58:58.586757 8111 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260616 19:58:58.599507 10488 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260616 19:58:58.604655 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 70058.6us
I20260616 19:58:58.610146 8817 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260616 19:58:58.609930 10488 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260616 19:58:58.640093 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 124203us
I20260616 19:58:58.659221 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260616 19:58:58.660296 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 95461.6us
I20260616 19:58:58.680190 7961 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260616 19:58:58.689801 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 100575us
I20260616 19:58:58.713397 10540 ts_tablet_manager.cc:933] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
W20260616 19:58:58.778568 8008 consensus_peers.cc:597] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7323d4e3a76f4800925df28a0f090072. This is attempt 1: this message will repeat every 5th retry.
W20260616 19:58:58.797885 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 96431.4us
W20260616 19:58:58.834236 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 43968.3us
I20260616 19:58:58.838449 8040 raft_consensus.cc:1275] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:58.800861 10554 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260616 19:58:58.825685 10510 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260616 19:58:58.897785 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 59074.9us
W20260616 19:58:58.910429 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 51261.9us
I20260616 19:58:58.959616 7968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:58:58.965957 7968 raft_consensus.cc:606] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:58:58.977308 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 66851.3us
W20260616 19:58:59.043089 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 47760.7us
W20260616 19:58:59.082582 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 99194.3us
W20260616 19:58:59.109468 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 195090us
I20260616 19:58:58.963608 10320 consensus_queue.cc:1048] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.104s
W20260616 19:58:59.209451 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 231234us
I20260616 19:58:59.231673 8111 raft_consensus.cc:1275] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260616 19:58:59.248112 10540 tablet_copy_client.cc:323] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
W20260616 19:58:59.308934 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 203367us
W20260616 19:58:59.529688 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 105653us
I20260616 19:58:59.431234 10545 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:58:59.545195 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 100470us
I20260616 19:58:59.547091 10583 ts_tablet_manager.cc:933] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
W20260616 19:58:59.599627 10567 log.cc:927] Time spent T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.512s user 0.001s sys 0.000s
I20260616 19:58:59.616721 10559 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:59.617420 10559 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.549474 10545 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.639262 10503 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:58:59.690574 10577 ts_tablet_manager.cc:933] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:59.709057 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 158963us
I20260616 19:58:59.686401 10499 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:59.576298 10543 consensus_queue.cc:1048] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.122s
I20260616 19:58:59.703042 10580 ts_tablet_manager.cc:933] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:58:59.694931 10572 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:58:59.716173 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 103234us
I20260616 19:58:59.696779 10503 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:58:59.829232 10585 log.cc:927] Time spent T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.134s user 0.001s sys 0.000s
W20260616 19:58:59.832613 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 97804.1us
W20260616 19:58:59.833679 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 94602.3us
I20260616 19:58:59.726475 10572 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:58:59.835139 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 100121us
I20260616 19:58:59.836102 10572 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:59.838761 10330 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:59.839191 10590 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:59.839321 10330 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.839731 10590 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:58:59.841895 10590 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:59.832690 10510 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:58:59.843008 10545 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:59.842924 10510 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.845160 10510 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:59.728302 10573 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:59.847708 10229 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:58:59.848181 10229 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.716748 10499 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.850070 10229 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:59.847272 10573 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.728469 10559 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:58:59.850828 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 121588us
I20260616 19:58:59.720055 10472 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:59.841897 10503 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:58:59.949659 10330 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:59.935943 10549 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
W20260616 19:58:59.983803 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 76620.8us
I20260616 19:58:59.988397 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:58:59.988750 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 254410us
I20260616 19:58:59.964908 10586 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:58:59.952347 10575 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:58:59.980262 10499 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:58:59.931303 10472 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:58:59.983953 10577 tablet_copy_client.cc:323] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.130:43051
I20260616 19:58:59.999651 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:59.998195 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:58:59.987488 10583 tablet_copy_client.cc:323] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
I20260616 19:59:00.025902 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.026276 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.026998 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.038116 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.040076 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:58:59.989837 10549 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:59.995975 10586 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:58:59.997200 10575 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:00.098953 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 57302.2us
W20260616 19:59:00.100497 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 61072.4us
W20260616 19:59:00.156541 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 156963us
W20260616 19:59:00.165870 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 64373.4us
I20260616 19:59:00.171151 10472 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.135399 10545 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:00.173931 10545 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:00.176200 10545 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:00.074841 10580 tablet_copy_client.cc:323] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Beginning tablet copy session from remote peer at address 127.6.159.129:42531
I20260616 19:59:00.134929 10596 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.165802 10330 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.192693 10330 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.193480 10586 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:00.195158 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:00.196612 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:00.160494 10590 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.169065 10597 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.202039 10330 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.187044 10549 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:00.227418 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41276 (request call id 22) took 1318 ms. Trace:
I20260616 19:59:00.231110 10573 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:00.227993 7968 rpcz_store.cc:276] 0616 19:58:58.909186 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:58.910325 (+ 1139us) service_pool.cc:224] Handling call
0616 19:59:00.210690 (+1300365us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b54000f3fa0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f262900 00007fe86f2dfa32 00007fe8742add0a 00007fe86d831934 00007fe86d8318c2 00007fe86d8317f1 00007fe86d83176c 00007fe86d830a22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:00.227053 (+ 16363us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b54000f3fa0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2609c7 00007fe86f2e0048 00007fe86f2dfc2a 00007fe8742add0a 00007fe86d831934 00007fe86d8318c2 00007fe86d8317f1 00007fe86d83176c 00007fe86d830a22 00007fe86d61dc3c
0616 19:59:00.227285 (+ 232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49818240}
W20260616 19:59:00.243122 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 85587.8us
I20260616 19:59:00.216320 10559 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:00.198797 10596 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.213261 10590 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:00.261308 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 159101us
I20260616 19:59:00.251909 10557 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:00.263455 7968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30950534b1834f6886db15f85f0abf14"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:59:00.266535 7968 raft_consensus.cc:606] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:00.253901 10560 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.216944 10597 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.270663 8055 tablet_copy_service.cc:140] P de4c5ce3ea8345ae9068f825b8d60e19: Received BeginTabletCopySession request for tablet 7323d4e3a76f4800925df28a0f090072 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:46718)
I20260616 19:59:00.258934 10575 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:00.276530 7979 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet f24cd55ee50d4bcfaee34b2adaa9b08a from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
I20260616 19:59:00.268391 10601 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.297091 10601 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.298130 7979 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet f24cd55ee50d4bcfaee34b2adaa9b08a from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-f24cd55ee50d4bcfaee34b2adaa9b08a
I20260616 19:59:00.299292 10601 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.300632 8055 tablet_copy_service.cc:161] P de4c5ce3ea8345ae9068f825b8d60e19: Beginning new tablet copy session on tablet 7323d4e3a76f4800925df28a0f090072 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:46718: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-7323d4e3a76f4800925df28a0f090072
I20260616 19:59:00.305035 7968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:59:00.280820 7980 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet 69c80791d0c44cb0b616bf402ae91b46 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
I20260616 19:59:00.305677 7968 raft_consensus.cc:606] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:00.306015 7980 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet 69c80791d0c44cb0b616bf402ae91b46 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-69c80791d0c44cb0b616bf402ae91b46
I20260616 19:59:00.268992 10557 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:00.306686 10590 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.308166 10557 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:00.282382 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.256500 10559 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:00.282469 10330 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:00.309624 10330 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.310218 10559 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:00.276897 7978 tablet_copy_service.cc:140] P b022cac8cb5e46b7a39a748e81e257a7: Received BeginTabletCopySession request for tablet b14a29bdfa9541db9d3f1e93a225a342 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 ({username='slave'} at 127.0.0.1:37398)
I20260616 19:59:00.310978 7978 tablet_copy_service.cc:161] P b022cac8cb5e46b7a39a748e81e257a7: Beginning new tablet copy session on tablet b14a29bdfa9541db9d3f1e93a225a342 from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 at {username='slave'} at 127.0.0.1:37398: session id = c06f6f0858fe4ce6bd5b55efeb5ba2d8-b14a29bdfa9541db9d3f1e93a225a342
I20260616 19:59:00.309960 10575 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:00.312287 10575 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.313941 10596 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.314239 8040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:59:00.314951 8040 raft_consensus.cc:606] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:00.284567 10560 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:00.317977 10560 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.311753 10330 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.314291 10575 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:00.319726 7968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:59:00.321843 7968 raft_consensus.cc:606] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:00.323170 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:59:00.282821 10597 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.324222 7978 tablet_copy_source_session.cc:215] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:59:00.329164 7979 tablet_copy_source_session.cc:215] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:59:00.333832 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.338975 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.344082 8117 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:00.470395 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:00.500252 10599 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:00.503505 10599 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.505614 10599 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:00.505689 10583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24cd55ee50d4bcfaee34b2adaa9b08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:59:00.354820 10605 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.508880 10605 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:00.541750 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 270863us
I20260616 19:59:00.545202 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:00.547626 8784 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s user 0.006s sys 0.016s
W20260616 19:59:00.548054 8784 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s user 0.006s sys 0.016s
W20260616 19:59:00.548321 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 516209us
I20260616 19:59:00.377065 10551 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.608876 7980 tablet_copy_source_session.cc:215] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:59:00.612396 8055 tablet_copy_source_session.cc:215] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: Tablet Copy: opened 0 blocks and 1 log segments
I20260616 19:59:00.573487 10551 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.615216 10551 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:00.647516 10614 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:00.659857 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.660903 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.675484 8037 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:00.672880 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.676510 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:59:00.677410 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:00.679150 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 196) took 1970 ms. Trace:
I20260616 19:59:00.679469 7865 rpcz_store.cc:276] 0616 19:58:58.708402 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:58.708598 (+ 196us) service_pool.cc:224] Handling call
0616 19:59:00.679096 (+1970498us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":600098,"reported_tablets":1}
I20260616 19:59:00.691918 10580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b14a29bdfa9541db9d3f1e93a225a342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:59:00.692930 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.693008 8009 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:00.693701 8045 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:00.694205 10391 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:00.694921 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.695609 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:00.696630 10391 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:00.696998 10391 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.697507 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.697352 10391 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:00.674842 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.704741 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.705859 10472 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.708214 10472 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.706697 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.709842 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.725955 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.710341 10472 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:00.711431 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.712898 10540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c80791d0c44cb0b616bf402ae91b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:59:00.714450 10584 raft_consensus.cc:993] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:00.715220 10561 raft_consensus.cc:993] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:00.722711 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.720009 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.751631 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.753190 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.768352 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.769219 10391 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.777195 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:00.739089 10387 raft_consensus.cc:1081] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:00.735060 10614 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.756098 10472 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.753894 10427 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:00.930833 8119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53916 (request call id 1618) took 1096 ms. Trace:
I20260616 19:59:00.779852 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.888652 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.849516 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 180) took 1478 ms. Trace:
I20260616 19:59:00.904393 10391 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:00.782511 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.951156 7966 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:00.953816 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:00.957615 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.955927 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.961329 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:00.965400 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.782312 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1545) took 1416 ms. Trace:
I20260616 19:59:00.819700 10584 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
W20260616 19:59:00.908277 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 127646us
I20260616 19:59:00.966459 7963 rpcz_store.cc:276] 0616 19:58:59.363458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:59.370568 (+ 7110us) service_pool.cc:224] Handling call
0616 19:59:00.779297 (+1408729us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:00.779491 (+ 194us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:00.779662 (+ 171us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:00.779687 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:00.779725 (+ 38us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:00.779756 (+ 31us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:00.779805 (+ 49us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:00.779910 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42657664}
I20260616 19:59:00.790374 10597 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260616 19:59:01.096678 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 95640.1us
I20260616 19:59:01.105144 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.110932 8084 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:01.114737 10229 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:01.115229 10229 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.115629 10229 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:00.835778 10587 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.933809 8119 rpcz_store.cc:276] 0616 19:58:59.834843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:59.835642 (+ 799us) service_pool.cc:224] Handling call
0616 19:58:59.835907 (+ 265us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:00.029008 (+193101us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:00.029037 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:00.029078 (+ 41us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:00.029108 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:00.930572 (+901464us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400102f54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:00.930628 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:00.930751 (+ 123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443668352}
I20260616 19:59:00.948962 10427 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.850155 10569 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:00.790064 10575 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:00.949723 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.142630 10575 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.143354 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.949904 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:00.950184 10558 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:00.778919 10561 raft_consensus.cc:1081] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:00.936684 7969 rpcz_store.cc:276] 0616 19:58:59.355777 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:59.365194 (+ 9417us) service_pool.cc:224] Handling call
0616 19:59:00.833652 (+1468458us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:00.833924 (+ 272us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:00.834152 (+ 228us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:00.834201 (+ 49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:00.834260 (+ 59us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:00.834303 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:00.834378 (+ 75us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:00.834517 (+ 139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17462144}
I20260616 19:59:00.976326 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.125646 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.094842 10597 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.097003 10566 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:01.066531 10472 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:01.142577 10569 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.143532 10229 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:00.849915 10387 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.147766 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.144661 8115 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:59:01.137965 10546 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:01.153693 10597 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:01.176083 8037 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:01.136920 10587 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.184103 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.178552 10229 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.159096 10566 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.198545 10566 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.201037 10614 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.205663 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7323d4e3a76f4800925df28a0f090072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260616 19:59:01.150223 10558 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:01.213323 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 84912.5us
I20260616 19:59:01.217288 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.163609 10387 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:01.215723 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 80677.8us
I20260616 19:59:01.223591 8110 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30950534b1834f6886db15f85f0abf14"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:59:01.223753 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.226467 8110 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:01.226840 8110 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:01.225126 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.224642 7934 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.231873 10564 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:01.172573 10546 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:01.237759 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:01.238299 8038 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:01.241142 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.219995 10575 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.219156 10566 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:01.244639 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.247516 10229 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.248816 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:01.234222 10488 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:01.254520 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.254446 10229 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.285885 10587 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.287631 10387 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:01.288802 10621 log.cc:927] Time spent T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.099s user 0.000s sys 0.000s
I20260616 19:59:01.288635 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.289335 10488 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.291476 10488 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.292497 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:01.293756 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.295518 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.297668 10546 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.297926 8085 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.299088 8085 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.306368 8038 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:01.313390 8110 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.322677 10229 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.322779 8110 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.323563 10229 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:59:01.324627 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 87720us
I20260616 19:59:01.250180 10566 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.320604 10601 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:01.321805 10427 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.330474 8085 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.327693 10564 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:01.409387 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:01.410465 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.411764 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.359342 8040 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:01.360466 10472 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.365957 10625 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:01.369161 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.372927 10569 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:01.386418 10583 consensus_meta.cc:287] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.597s user 0.010s sys 0.000s
I20260616 19:59:01.423971 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:01.392191 10580 consensus_meta.cc:287] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.507s user 0.003s sys 0.003s
I20260616 19:59:01.393550 10566 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.394239 10558 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:01.395370 10618 log.cc:927] Time spent T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.245s user 0.000s sys 0.003s
I20260616 19:59:01.400416 10605 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.404765 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.406553 10540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52474 (request call id 126) took 3102 ms. Trace:
I20260616 19:59:01.440286 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.415400 7867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55744 (request call id 217) took 2433 ms. Trace:
I20260616 19:59:01.416529 8044 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:59:01.418710 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 88350us
I20260616 19:59:01.424810 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2141) took 1450 ms. Trace:
I20260616 19:59:01.441493 7867 rpcz_store.cc:276] 0616 19:58:59.036721 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:59.109584 (+ 72863us) service_pool.cc:224] Handling call
0616 19:59:01.406692 (+2297108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":753761,"reported_tablets":3}
I20260616 19:59:01.441987 8085 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.426318 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:01.422370 7967 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:59:01.441944 7964 rpcz_store.cc:276] 0616 19:58:59.974637 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:00.928656 (+954019us) service_pool.cc:224] Handling call
0616 19:59:00.929062 (+ 406us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:01.153368 (+224306us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:01.153411 (+ 43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:01.153469 (+ 58us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:01.153513 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:01.420522 (+267009us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54003fe654. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:01.420570 (+ 48us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:01.424753 (+ 4183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515319680}
I20260616 19:59:01.445436 7970 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:59:01.446064 8085 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.447463 8084 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:01.449426 10581 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:01.438725 10540 rpcz_store.cc:276] 0616 19:58:58.304025 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:58.304190 (+ 165us) service_pool.cc:224] Handling call
0616 19:58:58.747468 (+443278us) ts_tablet_manager.cc:934] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
0616 19:59:00.711641 (+1964173us) tablet_copy_client.cc:368] Tablet copy session begun
0616 19:59:01.297412 (+585771us) tablet_metadata.cc:665] Metadata flushed
0616 19:59:01.297526 (+ 114us) tablet_copy_client.cc:497] Wrote new tablet metadata
0616 19:59:01.405784 (+108258us) tablet_copy_client.cc:418] Wrote new consensus metadata
0616 19:59:01.406080 (+ 296us) tablet_replica.cc:206] Creating consensus instance
0616 19:59:01.406389 (+ 309us) ts_tablet_manager.cc:989] Replica registered.
0616 19:59:01.406456 (+ 67us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":67860,"tablet-copy.queue_time_us":372997,"thread_start_us":349866,"threads_started":2}
I20260616 19:59:01.458384 10540 tablet_copy_client.cc:806] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:59:01.459125 10540 tablet_copy_client.cc:670] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:59:01.462627 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52474 (request call id 129) took 2550 ms. Trace:
I20260616 19:59:01.427227 10625 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.472890 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52474 (request call id 130) took 2404 ms. Trace:
I20260616 19:59:01.416283 10564 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.479527 8084 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:01.430104 10472 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:59:01.483193 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 154861us
I20260616 19:59:01.488692 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.490900 7934 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.492468 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.493335 10551 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.493892 10551 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.494146 10627 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:01.494395 10566 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260616 19:59:01.494688 10627 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.495155 10627 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:01.495236 10581 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:01.495880 10581 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:01.496769 10566 raft_consensus.cc:993] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:01.497008 10587 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.497260 10566 raft_consensus.cc:1081] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:01.497553 10587 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:01.496807 10581 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.501029 10597 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.477581 10605 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:01.501749 8085 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.503073 10597 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.503696 10597 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:01.505831 10427 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:01.506356 10427 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.510241 8044 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:01.508312 10427 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.511520 10564 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.512545 10625 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.504074 10597 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:01.512758 10565 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.463193 10583 rpcz_store.cc:276] 0616 19:58:58.971142 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:58.974084 (+ 2942us) service_pool.cc:224] Handling call
0616 19:58:59.592386 (+618302us) ts_tablet_manager.cc:934] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
0616 19:59:00.469040 (+876654us) tablet_copy_client.cc:368] Tablet copy session begun
0616 19:59:00.684502 (+215462us) tablet_metadata.cc:665] Metadata flushed
0616 19:59:00.706036 (+ 21534us) tablet_copy_client.cc:497] Wrote new tablet metadata
0616 19:59:01.431268 (+725232us) tablet_copy_client.cc:418] Wrote new consensus metadata
0616 19:59:01.462092 (+ 30824us) tablet_replica.cc:206] Creating consensus instance
0616 19:59:01.462472 (+ 380us) ts_tablet_manager.cc:989] Replica registered.
0616 19:59:01.462552 (+ 80us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":1433,"tablet-copy.queue_time_us":396081,"thread_start_us":500280,"threads_started":2}
I20260616 19:59:01.515259 10587 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:01.493063 8036 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:59:01.503818 8085 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.520239 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:01.521596 7934 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:01.525550 8084 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:01.473560 10580 rpcz_store.cc:276] 0616 19:58:59.213088 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:59.219056 (+ 5968us) service_pool.cc:224] Handling call
0616 19:58:59.721272 (+502216us) ts_tablet_manager.cc:934] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
0616 19:59:00.690623 (+969351us) tablet_copy_client.cc:368] Tablet copy session begun
0616 19:59:00.825211 (+134588us) tablet_metadata.cc:665] Metadata flushed
0616 19:59:00.825376 (+ 165us) tablet_copy_client.cc:497] Wrote new tablet metadata
0616 19:59:01.438875 (+613499us) tablet_copy_client.cc:418] Wrote new consensus metadata
0616 19:59:01.464711 (+ 25836us) tablet_replica.cc:206] Creating consensus instance
0616 19:59:01.472685 (+ 7974us) ts_tablet_manager.cc:989] Replica registered.
0616 19:59:01.472790 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":42488,"mutex_wait_us":2247,"tablet-copy.queue_time_us":441361,"thread_start_us":181427,"threads_started":2}
I20260616 19:59:01.532302 8085 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.534582 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.537297 8085 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:01.518931 10551 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.536219 10583 tablet_copy_client.cc:806] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:59:01.559298 10583 tablet_copy_client.cc:670] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:59:01.538177 10634 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:01.558698 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.539245 10419 raft_consensus.cc:993] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:01.561163 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.561771 10419 raft_consensus.cc:1081] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:01.530606 10565 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.539383 10628 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:01.539855 10580 tablet_copy_client.cc:806] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:59:01.565069 10580 tablet_copy_client.cc:670] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:59:01.538380 10625 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:01.540902 10601 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:01.567404 10601 raft_consensus.cc:697] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:01.531807 10406 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:01.541992 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:01.543568 7970 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:01.569211 7970 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:01.543612 8085 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:01.529800 10612 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.572508 10612 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:01.574779 10612 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:01.574905 10601 consensus_queue.cc:237] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.528561 10597 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:01.575497 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.577633 10565 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.547758 8084 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:01.549345 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.584343 8085 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.552688 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.553716 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.554383 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.572515 10625 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:59:01.593569 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.594187 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.586189 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.588572 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.590109 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.590785 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.591840 8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:59:01.596222 8115 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:01.596468 8085 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.593181 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.594187 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.596611 8115 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:01.597607 8085 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.545303 10627 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.593101 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.595814 7970 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:01.601554 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.599457 8085 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.604893 8085 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:01.605371 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 107504us
I20260616 19:59:01.606914 8036 raft_consensus.cc:1240] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:01.609356 10582 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:01.611101 8011 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:01.612311 8011 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:01.614063 10488 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:01.614718 10488 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:01.615006 10627 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.600944 10406 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:59:01.615012 10597 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.616052 10406 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.538707 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:46712 (request call id 4) took 3249 ms. Trace:
I20260616 19:59:01.629156 7970 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:01.593034 10628 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:01.617722 8011 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:01.638358 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:01.639009 7963 raft_consensus.cc:3055] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:01.639362 7963 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:01.632345 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:01.633466 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.640340 8111 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:01.628572 8116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:59:01.641403 8116 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:01.641866 8116 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:01.634104 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:01.642133 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.617718 8115 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.634793 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:01.643945 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.630295 8784 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.593791 10625 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:01.600517 10605 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:01.598603 10634 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:01.603895 10472 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:01.659423 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.647524 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:01.645979 7963 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.648015 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.606783 10563 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:01.654575 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:01.656333 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:01.657939 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.658459 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:01.664479 8115 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.633585 10406 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:01.667323 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.667894 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:01.672834 8044 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:01.668044 8111 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:01.668747 10583 tablet_copy_client.cc:538] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:59:01.670053 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.675220 8113 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:01.683693 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.685139 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.685564 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:01.687829 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:01.688872 7963 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:01.649904 10625 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:01.650307 10587 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:59:01.711982 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 51890.7us
I20260616 19:59:01.669518 10626 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:01.667061 10564 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:01.667299 10630 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:01.730077 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:01.700047 10540 tablet_copy_client.cc:538] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:59:01.701223 10580 tablet_copy_client.cc:538] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260616 19:59:01.731423 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 69132us
I20260616 19:59:01.703791 10600 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:59:01.735067 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 61340.3us
I20260616 19:59:01.742555 8011 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:01.713371 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:01.746168 8044 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:01.755101 7963 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:01.755404 8116 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:01.736109 10387 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:01.772097 7935 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.773778 7935 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.778729 10564 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:01.791944 8115 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:01.762727 10569 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:01.788580 8084 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:01.794860 10558 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:59:01.822149 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 47185.8us
W20260616 19:59:01.821806 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 88132.5us
I20260616 19:59:01.775501 10622 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260616 19:59:01.823932 8085 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.839304 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.817206 10627 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:01.844750 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.851315 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:01.856581 8085 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.859172 10582 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:01.860170 10564 raft_consensus.cc:993] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:59:01.632813 10584 rpcz_store.cc:276] 0616 19:58:58.285068 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:58.286467 (+ 1399us) service_pool.cc:224] Handling call
0616 19:58:58.310276 (+ 23809us) log.cc:844] Serialized 506 byte log entry
0616 19:58:58.746886 (+436610us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4c002546c0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86d5af120 00007fe86d5c0129 00007fe86d5c083f 00007fe86d5849ab 00007fe86d5b263c 00007fe86d5b2afe 00007fe86d84066b 00007fe86f241761 00007fe86f23f64b 00007fe86f246133 00007fe86f2460b9
0616 19:59:01.532853 (+2785967us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":205,"mutex_wait_us":32601,"raft.queue_time_us":8851,"raft.run_cpu_time_us":18051,"raft.run_wall_time_us":325385,"spinlock_wait_cycles":93314304,"thread_start_us":99511,"threads_started":1,"wal-append.queue_time_us":218047,"wal-append.run_cpu_time_us":1665,"wal-append.run_wall_time_us":1011906}
I20260616 19:59:01.860675 10647 raft_consensus.cc:1081] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:59:01.844105 10645 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:01.862388 7934 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:01.868253 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.838361 10597 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.855057 10577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52470 (request call id 91) took 2754 ms. Trace:
I20260616 19:59:01.969158 10577 rpcz_store.cc:276] 0616 19:58:59.101106 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:58:59.101284 (+ 178us) service_pool.cc:224] Handling call
0616 19:58:59.702872 (+601588us) ts_tablet_manager.cc:934] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Initiating tablet copy from peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
0616 19:59:01.162999 (+1460127us) tablet_copy_client.cc:368] Tablet copy session begun
0616 19:59:01.469269 (+306270us) tablet_metadata.cc:665] Metadata flushed
0616 19:59:01.469435 (+ 166us) tablet_copy_client.cc:497] Wrote new tablet metadata
0616 19:59:01.776811 (+307376us) tablet_copy_client.cc:418] Wrote new consensus metadata
0616 19:59:01.813361 (+ 36550us) tablet_replica.cc:206] Creating consensus instance
0616 19:59:01.845317 (+ 31956us) ts_tablet_manager.cc:989] Replica registered.
0616 19:59:01.853357 (+ 8040us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":47476,"tablet-copy.queue_time_us":502459,"thread_start_us":466535,"threads_started":2}
I20260616 19:59:01.970709 10577 tablet_copy_client.cc:806] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 0 data blocks...
I20260616 19:59:01.971243 10577 tablet_copy_client.cc:670] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Starting download of 1 WAL segments...
I20260616 19:59:01.881615 10645 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:01.882848 10636 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:01.898867 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.974798 10636 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:01.975126 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 91123.8us
I20260616 19:59:01.976286 7935 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.899662 10561 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:01.900375 10647 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:01.855329 10605 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:01.857559 10575 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:01.902545 10622 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:01.907748 10406 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:01.916405 10627 raft_consensus.cc:697] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:01.998847 10406 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.998937 10627 consensus_queue.cc:237] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:01.999569 8111 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
W20260616 19:59:01.922894 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 53581.6us
W20260616 19:59:01.923365 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 65616.1us
I20260616 19:59:01.942246 8116 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:02.005105 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.947441 10580 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:59:01.955442 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:02.013019 8820 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:01.951207 10625 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:01.958588 10650 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:01.943476 10583 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:59:02.022256 7969 raft_consensus.cc:1240] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 2. Current term is 3. Ops: []
I20260616 19:59:02.023802 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:02.024140 8115 raft_consensus.cc:1240] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:02.024430 7970 raft_consensus.cc:3055] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:02.024741 7970 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:01.935976 7866 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:01.946008 7865 catalog_manager.cc:5696] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 7 to 8, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 8 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:59:01.975765 10636 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:01.982260 10566 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:01.983721 10652 consensus_queue.cc:1059] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:01.985118 7935 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:01.985687 10561 consensus_queue.cc:1243] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260616 19:59:01.986330 10645 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:01.988453 10648 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:02.006955 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:02.061071 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44074.6us
I20260616 19:59:02.014384 10229 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:02.019464 10625 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:02.061193 8041 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:02.011483 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:02.066237 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:02.025381 7970 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.021808 10650 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:02.182356 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 197292us
I20260616 19:59:02.189699 10523 consensus_queue.cc:1059] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260616 19:59:02.081561 10587 consensus_queue.cc:1059] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.115s
I20260616 19:59:02.196756 10523 raft_consensus.cc:3055] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:02.197137 10523 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:02.197394 10587 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:02.197952 10587 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:02.198393 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:02.198114 10523 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.080870 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:02.199332 10523 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:02.196215 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:02.147475 8041 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:02.202135 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:02.105265 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:02.204084 8036 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 7.
I20260616 19:59:02.194926 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:02.207125 8043 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:59:02.207403 7966 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:02.207720 8043 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:02.207762 7966 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:02.208244 8043 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:02.208412 8039 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:59:02.198843 10587 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:02.219079 8118 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:02.222119 8084 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:02.222915 10229 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:59:02.223412 10229 raft_consensus.cc:697] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:02.205945 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:02.228403 8011 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:02.130497 8045 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:59:02.129447 8820 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:02.237646 8038 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:59:02.237936 7967 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:59:02.239233 8043 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.241051 7970 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:02.224263 10229 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:02.253319 10488 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:02.256235 8082 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:02.262517 7970 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:02.133953 10577 tablet_copy_client.cc:538] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260616 19:59:02.254897 7966 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:02.106976 10558 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:02.133808 10540 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:59:02.272313 10636 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260616 19:59:02.298046 10606 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:02.298288 10628 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:59:02.082441 10646 raft_consensus.cc:3055] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:02.322872 8043 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:02.333319 8115 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:02.332350 7966 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:59:02.330204 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 57454us
I20260616 19:59:02.338353 8085 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:02.339516 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 111227us
I20260616 19:59:02.340883 8011 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:02.188562 10644 consensus_queue.cc:1059] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.133s
I20260616 19:59:02.354862 8011 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:02.359269 8011 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:02.337395 10651 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:02.324961 10623 consensus_queue.cc:1059] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260616 19:59:02.339238 10646 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:02.373926 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:02.374594 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:02.365495 10544 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:02.386899 7964 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:02.391494 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:02.392206 8117 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:02.373061 10565 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260616 19:59:02.393303 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:02.372200 10584 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:02.393909 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:02.081369 10650 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.403841 7961 raft_consensus.cc:3055] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:02.406286 7961 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:02.407054 7961 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.378695 10611 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:02.441025 8785 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:02.441952 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 85729.3us
I20260616 19:59:02.443032 10558 raft_consensus.cc:3055] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:59:02.405727 10575 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260616 19:59:02.405871 10581 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260616 19:59:02.464313 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:02.464658 10561 raft_consensus.cc:3055] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:02.471421 10561 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:02.433871 10646 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.464691 10588 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260616 19:59:02.443452 10558 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:02.472688 10561 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.449524 7966 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:02.477548 10561 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:02.481156 10647 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:02.496462 10647 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:02.497208 10650 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:02.497393 10647 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.498690 10647 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:02.482625 10600 raft_consensus.cc:3055] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:59:02.486069 10587 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:02.490051 10644 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:02.508615 8117 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
W20260616 19:59:02.502043 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 110348us
W20260616 19:59:02.503685 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 85688.3us
I20260616 19:59:02.473265 10646 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:02.523857 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 155412us
W20260616 19:59:02.528427 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 138061us
I20260616 19:59:02.500970 10600 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:02.506757 10644 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:02.531872 10651 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:02.532974 10558 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:02.537572 10644 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:02.548708 10600 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:02.558238 10600 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:02.552001 10651 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:02.594136 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260616 19:59:02.594256 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 64859.3us
I20260616 19:59:02.606060 8009 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:02.552771 10558 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:59:02.601563 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:02.601974 7963 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:59:02.621026 10556 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:02.622088 10556 raft_consensus.cc:697] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:02.604425 7961 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:59:02.634048 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 102902us
W20260616 19:59:02.620759 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 208299us
I20260616 19:59:02.646498 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:02.647827 10584 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:02.650823 8011 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:02.641122 7933 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:02.630884 10556 consensus_queue.cc:237] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:02.663218 10566 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260616 19:59:02.684352 10668 log.cc:927] Time spent T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Append to log took a long time: real 0.114s user 0.001s sys 0.000s
W20260616 19:59:02.721421 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:02.735692 10229 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:02.737764 7961 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
W20260616 19:59:02.776670 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 126149us
W20260616 19:59:02.784437 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:02.795331 8082 consensus_peers.cc:597] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:59:02.795331 10665 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260616 19:59:02.835120 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 44394us
W20260616 19:59:02.845355 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:02.845815 10665 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:02.846670 10644 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:02.846894 10665 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:02.882113 8818 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:02.883646 8111 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260616 19:59:02.890205 8039 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:59:02.892931 10648 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:59:02.893800 8044 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:02.879004 10577 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap starting.
I20260616 19:59:02.916502 8117 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
W20260616 19:59:02.922374 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 47995.6us
W20260616 19:59:02.925935 7933 consensus_peers.cc:597] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 -> Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343): Couldn't send request to peer c06f6f0858fe4ce6bd5b55efeb5ba2d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260616 19:59:02.932240 10626 consensus_queue.cc:1059] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260616 19:59:02.933720 10626 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:02.940582 10583 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:59:02.941829 10583 tablet_bootstrap.cc:492] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:59:02.942726 10583 ts_tablet_manager.cc:1403] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 1.000s user 0.030s sys 0.014s
I20260616 19:59:02.934159 10626 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:02.951329 10644 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:02.952716 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:02.953996 10665 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:02.966439 10626 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:02.972175 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:02.987069 10608 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:02.991501 10626 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:03.067787 10606 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:03.068534 10665 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:03.068575 10606 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:03.069362 10665 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:03.072270 10606 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:03.089900 10583 raft_consensus.cc:359] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:03.091466 10583 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:59:03.071904 10665 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:03.092547 10583 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:03.125980 10583 ts_tablet_manager.cc:1434] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.183s user 0.003s sys 0.009s
I20260616 19:59:03.145820 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:03.150974 8038 raft_consensus.cc:1240] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:03.155591 8037 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260616 19:59:03.158926 8818 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 2 LEARNER]: Advancing to term 3
I20260616 19:59:03.170203 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:03.171885 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:03.180298 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:03.181866 7979 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-f24cd55ee50d4bcfaee34b2adaa9b08a received from {username='slave'} at 127.0.0.1:37398
I20260616 19:59:03.182508 7979 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-f24cd55ee50d4bcfaee34b2adaa9b08a on tablet f24cd55ee50d4bcfaee34b2adaa9b08a with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:03.184643 7964 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260616 19:59:03.192998 8817 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260616 19:59:03.207309 7970 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:59:03.210893 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 224) took 1278 ms. Trace:
I20260616 19:59:03.216324 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55744 (request call id 218) took 1450 ms. Trace:
I20260616 19:59:03.225258 7867 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 7, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:03.245612 7966 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:03.212728 7866 rpcz_store.cc:276] 0616 19:59:01.931956 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:01.932089 (+ 133us) service_pool.cc:224] Handling call
0616 19:59:03.210659 (+1278570us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:02.557401 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:02.558068 (+ 667us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:02.558095 (+ 27us) write_op.cc:435] Acquired schema lock
0616 19:59:02.558110 (+ 15us) tablet.cc:662] Decoding operations
0616 19:59:02.558438 (+ 328us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:02.558488 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:02.558514 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:59:02.558710 (+ 196us) tablet.cc:701] Row locks acquired
0616 19:59:02.558726 (+ 16us) write_op.cc:260] PREPARE: finished
0616 19:59:02.558861 (+ 135us) write_op.cc:270] Start()
0616 19:59:02.558962 (+ 101us) write_op.cc:276] Timestamp: P: 1781639942558813 usec, L: 0
0616 19:59:02.558977 (+ 15us) op_driver.cc:348] REPLICATION: starting
0616 19:59:02.559615 (+ 638us) log.cc:844] Serialized 599 byte log entry
0616 19:59:02.976612 (+416997us) op_driver.cc:464] REPLICATION: finished
0616 19:59:03.156411 (+179799us) write_op.cc:301] APPLY: starting
0616 19:59:03.156526 (+ 115us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:03.156638 (+ 112us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:03.156772 (+ 134us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:03.157028 (+ 256us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:03.157190 (+ 162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:59:03.157232 (+ 42us) write_op.cc:312] APPLY: finished
0616 19:59:03.158666 (+ 1434us) log.cc:844] Serialized 23 byte log entry
0616 19:59:03.158837 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:03.158947 (+ 110us) write_op.cc:454] Released schema lock
0616 19:59:03.158985 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:03.159315 (+ 330us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:03.159347 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":179539,"apply.run_cpu_time_us":1823,"apply.run_wall_time_us":2883,"num_ops":1,"prepare.queue_time_us":470888,"prepare.run_cpu_time_us":2344,"prepare.run_wall_time_us":2441,"replication_time_us":417449,"thread_start_us":99901,"threads_started":1}]]}
I20260616 19:59:03.217433 7865 rpcz_store.cc:276] 0616 19:59:01.775058 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:01.778641 (+ 3583us) service_pool.cc:224] Handling call
0616 19:59:03.216053 (+1437412us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:02.445375 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:02.452860 (+ 7485us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:02.452914 (+ 54us) write_op.cc:435] Acquired schema lock
0616 19:59:02.452943 (+ 29us) tablet.cc:662] Decoding operations
0616 19:59:02.453372 (+ 429us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:02.453448 (+ 76us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:02.453487 (+ 39us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:59:02.460709 (+ 7222us) tablet.cc:701] Row locks acquired
0616 19:59:02.460741 (+ 32us) write_op.cc:260] PREPARE: finished
0616 19:59:02.478320 (+ 17579us) write_op.cc:270] Start()
0616 19:59:02.503222 (+ 24902us) write_op.cc:276] Timestamp: P: 1781639942476352 usec, L: 0
0616 19:59:02.505364 (+ 2142us) op_driver.cc:348] REPLICATION: starting
0616 19:59:02.552664 (+ 47300us) log.cc:844] Serialized 541 byte log entry
0616 19:59:02.933550 (+380886us) op_driver.cc:464] REPLICATION: finished
0616 19:59:03.141010 (+207460us) write_op.cc:301] APPLY: starting
0616 19:59:03.145473 (+ 4463us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:03.145640 (+ 167us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:03.145809 (+ 169us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:03.146109 (+ 300us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:03.146387 (+ 278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:59:03.146430 (+ 43us) write_op.cc:312] APPLY: finished
0616 19:59:03.146748 (+ 318us) log.cc:844] Serialized 23 byte log entry
0616 19:59:03.146955 (+ 207us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:03.147056 (+ 101us) write_op.cc:454] Released schema lock
0616 19:59:03.147117 (+ 61us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:03.147617 (+ 500us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:03.147652 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":32243,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":181110,"apply.run_cpu_time_us":2418,"apply.run_wall_time_us":19372,"mutex_wait_us":8972,"num_ops":1,"prepare.queue_time_us":364131,"prepare.run_cpu_time_us":6789,"prepare.run_wall_time_us":132429,"raft.queue_time_us":311981,"raft.run_cpu_time_us":89023,"raft.run_wall_time_us":151399,"replication_time_us":419212,"thread_start_us":510647,"threads_started":4,"wal-append.queue_time_us":16801}]]}
I20260616 19:59:03.253266 10540 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260616 19:59:03.268746 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 73594.7us
I20260616 19:59:03.268944 10406 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260616 19:59:03.269459 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 68536.7us
I20260616 19:59:03.369045 10540 tablet_bootstrap.cc:492] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:59:03.277500 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:03.282215 10614 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:03.288077 10580 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260616 19:59:03.375063 10580 tablet_bootstrap.cc:492] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:59:03.376199 10580 ts_tablet_manager.cc:1403] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 1.429s user 0.070s sys 0.036s
I20260616 19:59:03.288447 8115 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:59:03.295125 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:03.296615 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:03.299212 7969 raft_consensus.cc:1275] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:59:03.274590 10611 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260616 19:59:03.388408 8038 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260616 19:59:03.421454 10587 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260616 19:59:03.425462 10540 ts_tablet_manager.cc:1403] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 1.292s user 0.044s sys 0.041s
I20260616 19:59:03.379781 10580 raft_consensus.cc:359] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:59:03.459209 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 51713.3us
I20260616 19:59:03.458894 10540 raft_consensus.cc:359] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:03.444095 10580 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:59:03.498409 10540 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:59:03.536471 10662 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260616 19:59:03.543782 7966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:59:03.548274 7966 raft_consensus.cc:606] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:03.548708 7966 raft_consensus.cc:614] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:03.545315 10580 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:03.552421 10580 ts_tablet_manager.cc:1434] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.176s user 0.008s sys 0.001s
W20260616 19:59:03.626744 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 189398us
I20260616 19:59:03.672204 8820 raft_consensus.cc:1217] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Deduplicated request from leader. Original: 8.7->[8.8-8.8] Dedup: 8.8->[]
I20260616 19:59:03.584913 10540 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:59:03.678427 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 184670us
I20260616 19:59:03.679503 10540 ts_tablet_manager.cc:1434] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.244s user 0.006s sys 0.002s
I20260616 19:59:03.679548 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:59:03.680333 7963 raft_consensus.cc:606] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:03.687219 7963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
I20260616 19:59:03.688079 7963 raft_consensus.cc:606] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:03.688999 7963 raft_consensus.cc:614] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:03.693868 10387 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:03.694438 10387 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:03.720506 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 271673us
W20260616 19:59:03.740284 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 101175us
I20260616 19:59:03.747565 8011 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:03.721163 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 191478us
I20260616 19:59:03.778419 8011 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:03.769584 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 47991.5us
I20260616 19:59:03.790480 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:03.778131 8786 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:03.819324 10657 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:03.819587 7980 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-69c80791d0c44cb0b616bf402ae91b46 received from {username='slave'} at 127.0.0.1:37398
I20260616 19:59:03.820226 7980 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-69c80791d0c44cb0b616bf402ae91b46 on tablet 69c80791d0c44cb0b616bf402ae91b46 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:03.820312 10657 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:03.821133 10657 consensus_queue.cc:1048] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:03.802242 10665 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:03.904256 7963 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260616 19:59:03.824473 7979 tablet_copy_service.cc:342] P b022cac8cb5e46b7a39a748e81e257a7: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-b14a29bdfa9541db9d3f1e93a225a342 received from {username='slave'} at 127.0.0.1:37398
I20260616 19:59:03.893404 10643 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election won for term 8
W20260616 19:59:03.958273 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 146719us
I20260616 19:59:03.964298 8044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:59:03.965102 8044 raft_consensus.cc:606] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:03.965853 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 169144us
I20260616 19:59:03.961632 7979 tablet_copy_service.cc:434] P b022cac8cb5e46b7a39a748e81e257a7: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-b14a29bdfa9541db9d3f1e93a225a342 on tablet b14a29bdfa9541db9d3f1e93a225a342 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:03.975441 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 158306us
I20260616 19:59:03.968498 10658 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:03.976423 10612 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:03.981786 10670 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:04.012950 10612 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.013197 10670 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:04.000361 10605 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:04.014385 10605 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:03.987432 10658 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.016021 10670 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.017982 10658 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.018296 10612 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:03.991148 10626 raft_consensus.cc:993] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:59:04.019855 10626 raft_consensus.cc:1081] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:59:04.023401 10566 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:04.023993 10566 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.030686 10566 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.018326 10387 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:04.080482 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 282590us
I20260616 19:59:04.068866 10577 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260616 19:59:04.087512 10626 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260616 19:59:04.099202 10605 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:04.116408 10719 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:04.116658 10712 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:04.117084 10719 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.117326 10712 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.118716 10577 tablet_bootstrap.cc:492] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Bootstrap complete.
I20260616 19:59:04.119544 10577 ts_tablet_manager.cc:1403] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent bootstrapping tablet: real 1.276s user 0.035s sys 0.070s
I20260616 19:59:04.119707 10712 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.119894 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260616 19:59:04.108001 10643 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:04.119405 10719 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:04.125913 10646 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:04.121726 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:59:04.131053 10646 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.139827 10646 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.140841 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.141631 7962 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 3.
I20260616 19:59:04.141745 10719 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260616 19:59:04.140954 10643 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:04.158074 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:04.153158 10694 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:04.160727 10723 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:04.160861 10694 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.161382 10723 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.162874 10723 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:04.163367 10694 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.174250 10387 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:04.174866 10387 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:04.177071 10387 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.169232 10665 raft_consensus.cc:993] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:04.186971 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.196336 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:04.215400 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.193689 10694 raft_consensus.cc:1081] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:04.259394 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.260268 7965 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:04.261121 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.261962 7968 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:04.269033 10577 raft_consensus.cc:359] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.270583 10577 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEARNER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Initialized, Role: LEARNER
I20260616 19:59:04.271286 10577 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:04.282752 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.282756 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:04.260280 10627 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:04.285220 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.286010 7961 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:04.261147 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:04.284771 8039 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:04.287806 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.288059 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.296353 10588 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:04.285586 10627 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.302541 8112 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:04.311146 10577 ts_tablet_manager.cc:1434] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent starting tablet: real 0.191s user 0.005s sys 0.003s
I20260616 19:59:04.312330 8041 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:04.324047 8118 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:04.324079 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.327270 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.341739 10724 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:04.342537 10724 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:04.334990 10588 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.357369 10627 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:04.358202 8115 raft_consensus.cc:2393] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:04.359053 10588 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:04.362759 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.363705 8113 raft_consensus.cc:2393] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:04.368777 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.371071 10724 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:04.373944 10723 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:04.376320 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.377009 7961 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:04.379089 10626 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:04.380686 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.381377 7969 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:04.391921 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:04.392585 7966 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:04.379685 10626 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.395080 8818 raft_consensus.cc:1217] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Deduplicated request from leader. Original: 9.5->[9.6-9.6] Dedup: 9.6->[]
I20260616 19:59:04.398443 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:04.399128 8039 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:04.399369 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:04.399938 8041 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:04.400695 8044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51876
I20260616 19:59:04.401427 8044 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:04.401786 8044 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:04.402051 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.402580 10626 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:04.404119 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 146987us
I20260616 19:59:04.406756 10727 raft_consensus.cc:1064] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:59:04.402660 8111 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:04.408087 8011 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:04.409703 10665 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:04.410142 10665 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.410606 10665 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:04.415398 8011 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:04.417150 8009 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:04.417495 10727 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:04.419634 7969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:59:04.420284 7969 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:04.420637 7969 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:04.420871 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.421548 7970 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:04.422401 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.422484 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.422986 7970 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:04.423110 7963 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:04.423902 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:04.436542 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.438800 7969 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.437238 8115 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:04.440922 7969 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.444746 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:04.446372 8038 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:04.445708 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:04.447321 8085 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:04.447328 8114 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:04.448364 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:04.449254 7934 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:04.451627 10700 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:04.452569 8085 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:04.453066 10699 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:04.453493 10699 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.453866 10699 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:04.454525 10694 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:04.454972 10694 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:04.455369 10694 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:04.456449 10717 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:04.456875 10717 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.457305 10717 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:04.452026 10700 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.458211 10700 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:04.457612 7865 catalog_manager.cc:5696] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 4 to 5, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 5 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:04.460409 7866 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 2 to 3, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:59:04.464320 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:04.465250 8115 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:04.459322 7865 catalog_manager.cc:5696] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 5 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
W20260616 19:59:04.480027 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 48867.3us
W20260616 19:59:04.481240 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 50537.5us
I20260616 19:59:04.483810 8011 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:04.498492 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:04.499396 8815 raft_consensus.cc:2368] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
I20260616 19:59:04.484773 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:04.457520 10718 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:04.501715 10718 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:04.502072 10718 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:04.505173 7865 catalog_manager.cc:5696] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 5 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:04.516446 7867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 198) took 1710 ms. Trace:
I20260616 19:59:04.436044 10657 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260616 19:59:04.528294 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 47133.3us
W20260616 19:59:04.528470 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 44760.9us
I20260616 19:59:04.536706 8011 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:04.529105 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:04.539614 7966 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:04.548843 10387 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:04.550326 10229 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:04.550720 10229 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.551011 10229 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:04.552340 8082 leader_election.cc:400] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Vote denied by peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343) with higher term. Message: Invalid argument: T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
I20260616 19:59:04.552812 8082 leader_election.cc:403] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:59:04.546973 8085 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:04.548194 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:04.553582 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:04.528110 10723 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.545981 10658 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:04.566879 8044 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:04.570782 8037 raft_consensus.cc:3055] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:04.573513 8041 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:04.571810 8037 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:04.577977 8044 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:04.587208 10702 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:04.587734 10702 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:04.587839 10472 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 4
I20260616 19:59:04.588181 10702 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:04.569161 10658 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.572996 10662 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:04.530540 7867 rpcz_store.cc:276] 0616 19:59:02.805689 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:02.805875 (+ 186us) service_pool.cc:224] Handling call
0616 19:59:04.516340 (+1710465us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:03.653722 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:03.665635 (+ 11913us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:03.665685 (+ 50us) write_op.cc:435] Acquired schema lock
0616 19:59:03.665705 (+ 20us) tablet.cc:662] Decoding operations
0616 19:59:03.666087 (+ 382us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:03.666173 (+ 86us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:03.666203 (+ 30us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:59:03.666491 (+ 288us) tablet.cc:701] Row locks acquired
0616 19:59:03.666509 (+ 18us) write_op.cc:260] PREPARE: finished
0616 19:59:03.666682 (+ 173us) write_op.cc:270] Start()
0616 19:59:03.666820 (+ 138us) write_op.cc:276] Timestamp: P: 1781639943666611 usec, L: 0
0616 19:59:03.666842 (+ 22us) op_driver.cc:348] REPLICATION: starting
0616 19:59:03.667526 (+ 684us) log.cc:844] Serialized 599 byte log entry
0616 19:59:04.081427 (+413901us) op_driver.cc:464] REPLICATION: finished
0616 19:59:04.269571 (+188144us) write_op.cc:301] APPLY: starting
0616 19:59:04.269686 (+ 115us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:04.269797 (+ 111us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:04.269911 (+ 114us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:04.270124 (+ 213us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:04.270291 (+ 167us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:59:04.270362 (+ 71us) write_op.cc:312] APPLY: finished
0616 19:59:04.270559 (+ 197us) log.cc:844] Serialized 23 byte log entry
0616 19:59:04.270729 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:04.270834 (+ 105us) write_op.cc:454] Released schema lock
0616 19:59:04.270877 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:04.271280 (+ 403us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:04.271316 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41378,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":136513,"apply.run_cpu_time_us":1691,"apply.run_wall_time_us":14950,"num_ops":1,"prepare.queue_time_us":409153,"prepare.run_cpu_time_us":2847,"prepare.run_wall_time_us":15668,"replication_time_us":399011,"thread_start_us":344108,"threads_started":2}]]}
I20260616 19:59:04.592350 10658 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:04.593708 10662 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:04.634199 8055 tablet_copy_service.cc:342] P de4c5ce3ea8345ae9068f825b8d60e19: Request end of tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-7323d4e3a76f4800925df28a0f090072 received from {username='slave'} at 127.0.0.1:46718
I20260616 19:59:04.634941 8055 tablet_copy_service.cc:434] P de4c5ce3ea8345ae9068f825b8d60e19: ending tablet copy session c06f6f0858fe4ce6bd5b55efeb5ba2d8-7323d4e3a76f4800925df28a0f090072 on tablet 7323d4e3a76f4800925df28a0f090072 with peer c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:04.637876 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:04.647382 10720 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:04.648097 10720 raft_consensus.cc:697] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:04.649817 7966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49182
W20260616 19:59:04.654660 10730 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.166s user 0.001s sys 0.000s
I20260616 19:59:04.660666 8115 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:04.629917 10662 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:04.665213 7966 raft_consensus.cc:606] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Received request to transfer leadership
I20260616 19:59:04.640173 10700 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.649602 10699 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.681779 10700 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.652635 10702 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.688894 10229 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.649020 10720 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:04.689989 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 152193us
I20260616 19:59:04.693632 8011 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:04.706929 8037 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.708979 8037 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:04.711524 10229 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:04.695636 10694 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.726923 10694 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.707985 10472 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343) with higher term. Message: Invalid argument: T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4.
I20260616 19:59:04.734035 7934 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:04.732976 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:04.749277 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:04.750202 7961 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:04.722726 10665 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.716092 10699 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.748400 10700 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:04.769474 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:04.770982 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 195643us
I20260616 19:59:04.781622 10700 raft_consensus.cc:697] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:04.761083 10717 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.816047 8037 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:04.834795 10665 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:04.841403 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 147847us
I20260616 19:59:04.841094 10700 consensus_queue.cc:237] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.844357 10708 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:04.844513 10702 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:04.844848 10708 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:04.845381 10708 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:59:04.849352 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 80514.5us
I20260616 19:59:04.851500 8110 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:04.853067 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:04.827122 10668 log.cc:927] Time spent T 00000000000000000000000000000000 P 9602378bb66e49bcae343c682b95adcb: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260616 19:59:04.855455 8118 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 7: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:04.847224 10662 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:04.860770 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 89082.3us
I20260616 19:59:04.861845 7961 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:04.863168 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:04.863842 8042 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:04.878051 10717 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:04.879117 8114 raft_consensus.cc:2368] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6.
I20260616 19:59:04.833395 10718 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.881458 8116 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:04.881935 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:04.882550 8815 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:04.887882 10611 raft_consensus.cc:1064] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:59:04.890350 10611 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:04.891253 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:04.892038 8117 raft_consensus.cc:2368] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:04.892985 8115 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260616 19:59:04.894065 10612 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:04.894461 10658 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.894575 10612 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.898136 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:04.899559 8116 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:04.993800 8044 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:04.997648 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 97213.7us
W20260616 19:59:04.998417 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:05.000805 8817 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260616 19:59:04.986361 10708 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:04.986964 8815 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:05.005748 10662 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:05.007160 8009 leader_election.cc:400] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:05.006580 10665 raft_consensus.cc:476] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Not starting pre-election -- already a leader
I20260616 19:59:04.971670 10696 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260616 19:59:05.009591 10612 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:05.011137 10741 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:05.016702 10658 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:05.019111 8009 leader_election.cc:403] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260616 19:59:05.018123 10750 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260616 19:59:05.027985 10612 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:05.030867 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:05.032326 10741 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.035522 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:05.038077 8044 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260616 19:59:05.038622 10556 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:05.039090 10556 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:05.045447 10749 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260616 19:59:05.047660 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.048293 8036 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:05.048444 10710 raft_consensus.cc:993] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: : Instructing follower 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:05.070900 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:05.069487 10741 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:05.082432 7935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.009s sys 0.011s
W20260616 19:59:05.082873 7935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s user 0.009s sys 0.011s
W20260616 19:59:05.083129 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 518086us
I20260616 19:59:05.083724 10702 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:05.083994 10700 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:05.084342 10702 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.084615 10700 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.086117 8815 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:05.086808 10700 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:05.087054 8818 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260616 19:59:05.088348 8820 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEARNER]: Advancing to term 7
I20260616 19:59:05.088792 10702 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:05.089816 10702 consensus_queue.cc:1059] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:05.091078 10698 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:05.091434 10698 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:05.092957 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.074067 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.098039 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:05.099170 10718 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:05.099269 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:05.101389 8036 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:05.068289 8008 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:05.105983 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.106714 7969 raft_consensus.cc:2368] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:05.110270 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.111039 8037 raft_consensus.cc:2368] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6.
I20260616 19:59:05.109711 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.112183 8037 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260616 19:59:05.103201 8820 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:59:05.114666 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.115415 7969 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:05.116285 8038 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:05.092224 10698 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:05.117745 10698 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:05.123562 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:05.130002 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:05.122710 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:05.135437 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.135979 7969 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:05.136757 10708 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:05.117977 10700 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260616 19:59:05.138741 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.139518 7967 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:05.144361 10558 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:05.221164 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:05.221961 8110 raft_consensus.cc:2368] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
I20260616 19:59:05.171561 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:05.150604 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.208636 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:05.217151 10558 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:05.226776 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.227494 7963 raft_consensus.cc:2368] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9.
I20260616 19:59:05.228348 10558 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:05.232470 10556 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:05.232880 10746 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.117s user 0.001s sys 0.000s
I20260616 19:59:05.236173 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:05.245091 7970 raft_consensus.cc:2368] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
I20260616 19:59:05.245888 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:05.237273 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:05.227748 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.208598 7964 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260616 19:59:05.112059 10612 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:05.255371 8045 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 7: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:05.256459 8039 raft_consensus.cc:2393] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 6: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:05.257321 8011 leader_election.cc:400] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:05.258584 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.260609 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.263288 7968 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260616 19:59:05.115729 10741 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:05.258980 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:05.265812 8038 raft_consensus.cc:2368] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9.
I20260616 19:59:05.264870 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 225) took 1371 ms. Trace:
I20260616 19:59:05.272368 7969 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:05.269752 8011 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:05.144397 10710 raft_consensus.cc:1081] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Signalling peer 6827184e3cef42c2ada15b2d31afee4b to start an election
I20260616 19:59:05.209756 10718 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:05.269551 7865 rpcz_store.cc:276] 0616 19:59:03.894572 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:03.904166 (+ 9594us) service_pool.cc:224] Handling call
0616 19:59:05.264809 (+1360643us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:04.653021 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:04.653581 (+ 560us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:04.653615 (+ 34us) write_op.cc:435] Acquired schema lock
0616 19:59:04.653632 (+ 17us) tablet.cc:662] Decoding operations
0616 19:59:04.654168 (+ 536us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:04.654223 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:04.654248 (+ 25us) tablet.cc:685] Acquiring locks for 3 operations
0616 19:59:04.654567 (+ 319us) tablet.cc:701] Row locks acquired
0616 19:59:04.654580 (+ 13us) write_op.cc:260] PREPARE: finished
0616 19:59:04.654722 (+ 142us) write_op.cc:270] Start()
0616 19:59:04.654835 (+ 113us) write_op.cc:276] Timestamp: P: 1781639944654667 usec, L: 0
0616 19:59:04.654855 (+ 20us) op_driver.cc:348] REPLICATION: starting
0616 19:59:04.712437 (+ 57582us) log.cc:844] Serialized 1317 byte log entry
0616 19:59:04.886513 (+174076us) op_driver.cc:464] REPLICATION: finished
0616 19:59:04.886824 (+ 311us) write_op.cc:301] APPLY: starting
0616 19:59:04.886899 (+ 75us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:04.887026 (+ 127us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:04.887131 (+ 105us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:04.887621 (+ 490us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:04.887799 (+ 178us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0616 19:59:04.887843 (+ 44us) write_op.cc:312] APPLY: finished
0616 19:59:04.888041 (+ 198us) log.cc:844] Serialized 35 byte log entry
0616 19:59:04.888229 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:04.888318 (+ 89us) write_op.cc:454] Released schema lock
0616 19:59:04.888358 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:04.888729 (+ 371us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:04.888757 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":120315,"reported_tablets":5,"spinlock_wait_cycles":25856,"child_traces":[["op",{"apply.queue_time_us":129,"apply.run_cpu_time_us":1857,"apply.run_wall_time_us":1856,"num_ops":3,"prepare.queue_time_us":103262,"prepare.run_cpu_time_us":3275,"prepare.run_wall_time_us":59782,"replication_time_us":231523}]]}
I20260616 19:59:05.281407 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:05.277702 7935 leader_election.cc:400] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6.
I20260616 19:59:05.285197 7935 leader_election.cc:403] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260616 19:59:05.281456 10556 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:05.288269 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:05.289990 7864 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b reported cstate change: term changed from 6 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:05.292346 10566 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6.
I20260616 19:59:05.293478 10739 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260616 19:59:05.307263 8041 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
W20260616 19:59:05.117748 10626 consensus_queue.cc:1175] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" responder_term: 8 status { last_received { term: 7 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260616 19:59:05.280468 10741 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:05.326836 10741 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:05.316392 10718 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:05.330865 10718 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.334074 8011 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:05.335103 10658 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:05.335378 8084 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:05.336844 10721 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:05.337364 10721 raft_consensus.cc:697] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:05.338770 8036 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260616 19:59:05.342360 8011 leader_election.cc:400] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
I20260616 19:59:05.342854 8011 leader_election.cc:403] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260616 19:59:05.342833 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:05.344518 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:05.351928 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:05.348191 10743 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:05.354722 10694 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:05.435490 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 91274us
I20260616 19:59:05.338287 10721 consensus_queue.cc:237] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:05.449915 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 93948.1us
I20260616 19:59:05.455194 8084 leader_election.cc:400] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9.
I20260616 19:59:05.455600 8084 leader_election.cc:403] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260616 19:59:05.450958 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 99159.6us
I20260616 19:59:05.476292 10719 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9.
W20260616 19:59:05.478328 10756 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.130s user 0.000s sys 0.000s
I20260616 19:59:05.478760 8011 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:05.483645 8011 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:05.484736 8085 leader_election.cc:400] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9.
I20260616 19:59:05.486372 8085 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:05.487483 10472 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:05.486536 10716 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:05.493957 8009 leader_election.cc:400] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
W20260616 19:59:05.504215 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:05.505419 7934 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:05.514868 7934 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:05.539283 7934 leader_election.cc:400] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6.
I20260616 19:59:05.483896 10740 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
I20260616 19:59:05.567109 7934 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:05.559585 8112 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:59:05.518926 10698 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:05.564254 10757 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:05.505235 10657 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:05.486215 10724 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:05.573885 10700 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:05.578454 10720 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260616 19:59:05.580835 7961 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
W20260616 19:59:05.581058 10754 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.084s user 0.002s sys 0.000s
I20260616 19:59:05.583137 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:59:05.611881 8112 raft_consensus.cc:606] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Received request to transfer leadership
I20260616 19:59:05.531080 10558 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:05.595551 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:05.599784 7968 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:59:05.636575 10558 raft_consensus.cc:697] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:05.642112 10472 raft_consensus.cc:697] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:05.658406 10600 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260616 19:59:05.703751 10759 log.cc:927] Time spent T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.113s user 0.001s sys 0.000s
I20260616 19:59:05.726706 10770 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:05.727545 10770 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.738097 10770 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:05.738729 10647 raft_consensus.cc:1064] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:59:05.737540 10739 raft_consensus.cc:2955] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Committing config change with OpId 8.9: config changed from index 8 to 9, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:05.747169 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:05.750633 8820 raft_consensus.cc:2955] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Committing config change with OpId 8.9: config changed from index 8 to 9, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:05.700280 10558 consensus_queue.cc:237] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.729158 10741 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:05.761487 10741 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.767454 10741 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:05.778546 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:05.731556 10472 consensus_queue.cc:237] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.811945 7961 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49182
I20260616 19:59:05.812755 7961 raft_consensus.cc:606] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Received request to transfer leadership
I20260616 19:59:05.826206 10664 raft_consensus.cc:993] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:05.836877 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:05.844606 10588 raft_consensus.cc:1081] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
W20260616 19:59:05.875627 10755 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.378s user 0.003s sys 0.000s
I20260616 19:59:05.874326 10770 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:59:05.882404 7855 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 7323d4e3a76f4800925df28a0f090072 already in progress [suppressed 3 similar messages]
I20260616 19:59:05.890689 7964 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:59:05.891448 7964 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:05.891811 7964 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:05.891897 7969 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260616 19:59:05.894395 10643 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:59:05.897186 10664 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:05.901930 10406 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:05.902410 10406 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:05.909101 8045 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:59:05.914916 8815 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:59:05.915450 10647 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 4, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:05.919373 7964 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:05.920830 7966 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:05.923447 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:05.940714 8011 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:05.959539 7964 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:05.968784 8118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:05.969683 8118 raft_consensus.cc:606] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:05.976478 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:05.978152 10406 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:05.978744 10763 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:05.989902 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:05.990742 8817 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:05.993335 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:05.999374 8045 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:06.007886 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55744 (request call id 219) took 1559 ms. Trace:
I20260616 19:59:06.009569 7864 rpcz_store.cc:276] 0616 19:59:04.448841 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:04.448996 (+ 155us) service_pool.cc:224] Handling call
0616 19:59:06.007825 (+1558829us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:05.590725 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:05.654264 (+ 63539us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:05.656070 (+ 1806us) write_op.cc:435] Acquired schema lock
0616 19:59:05.657321 (+ 1251us) tablet.cc:662] Decoding operations
0616 19:59:05.688188 (+ 30867us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:05.701671 (+ 13483us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:05.703213 (+ 1542us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:59:05.717517 (+ 14304us) tablet.cc:701] Row locks acquired
0616 19:59:05.717541 (+ 24us) write_op.cc:260] PREPARE: finished
0616 19:59:05.717698 (+ 157us) write_op.cc:270] Start()
0616 19:59:05.717825 (+ 127us) write_op.cc:276] Timestamp: P: 1781639945717650 usec, L: 0
0616 19:59:05.717850 (+ 25us) op_driver.cc:348] REPLICATION: starting
0616 19:59:05.718535 (+ 685us) log.cc:844] Serialized 536 byte log entry
0616 19:59:05.807817 (+ 89282us) op_driver.cc:464] REPLICATION: finished
0616 19:59:05.904620 (+ 96803us) write_op.cc:301] APPLY: starting
0616 19:59:05.904699 (+ 79us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:05.904800 (+ 101us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:05.904937 (+ 137us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:05.905634 (+ 697us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:05.905788 (+ 154us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:59:05.905828 (+ 40us) write_op.cc:312] APPLY: finished
0616 19:59:05.906006 (+ 178us) log.cc:844] Serialized 23 byte log entry
0616 19:59:05.906177 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:05.906255 (+ 78us) write_op.cc:454] Released schema lock
0616 19:59:05.906291 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:05.906589 (+ 298us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:05.906620 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46545,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":96554,"apply.run_cpu_time_us":1967,"apply.run_wall_time_us":1967,"num_ops":1,"prepare.queue_time_us":230115,"prepare.run_cpu_time_us":3685,"prepare.run_wall_time_us":151704,"replication_time_us":89794,"thread_start_us":102283,"threads_started":2}]]}
I20260616 19:59:06.013166 7865 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 7 to 8, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 8 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:06.015077 7865 catalog_manager.cc:5696] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 7 to 8, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 8 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:06.016076 10606 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260616 19:59:06.033289 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 47739.4us
I20260616 19:59:06.028177 8045 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:06.039093 8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:53918
I20260616 19:59:06.041939 8117 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:06.042279 8117 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:06.048659 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 58201.7us
I20260616 19:59:06.053534 10229 raft_consensus.cc:993] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:06.056859 10229 raft_consensus.cc:1081] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:05.941234 10472 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260616 19:59:06.065058 7970 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:59:06.066157 8117 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:06.068889 8118 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:59:06.069386 8114 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:06.074719 8117 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:06.077571 8084 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:06.081925 10229 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:06.082872 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:06.084546 10657 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260616 19:59:06.079411 10775 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.219s user 0.001s sys 0.000s
I20260616 19:59:06.023572 10566 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260616 19:59:06.038631 7855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 187728us
I20260616 19:59:06.096475 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:06.101364 8043 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:06.097028 8084 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:06.099721 8817 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:06.103319 10229 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:06.109817 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:06.113528 7963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:59:06.114111 7963 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:06.114468 7963 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:06.119678 7933 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:06.120952 8044 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:59:06.122252 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:06.127566 7968 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:06.127839 7968 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:06.132292 8045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:59:06.133006 8045 raft_consensus.cc:606] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:06.128543 7968 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:06.100894 10406 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260616 19:59:06.144037 10727 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:06.142146 10702 consensus_queue.cc:1059] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260616 19:59:06.140031 10698 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260616 19:59:06.142511 10762 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
W20260616 19:59:06.185292 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 54390.7us
W20260616 19:59:06.187444 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 61260.3us
I20260616 19:59:06.189507 7963 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:06.191435 7963 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:06.192492 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:06.201762 8110 raft_consensus.cc:3055] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:06.194351 7969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49182
I20260616 19:59:06.202098 8110 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:06.194067 8117 raft_consensus.cc:2955] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Committing config change with OpId 8.9: config changed from index 8 to 9, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:06.202950 8110 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:06.204046 8110 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:06.186960 7966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:41276
W20260616 19:59:06.195506 10781 log.cc:927] Time spent T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
W20260616 19:59:06.196105 10784 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.097s user 0.001s sys 0.000s
I20260616 19:59:06.174619 10727 raft_consensus.cc:697] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:06.202085 8043 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:06.211992 10727 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:06.215137 8818 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:06.227859 8110 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:06.231583 7934 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:06.249125 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 61066us
I20260616 19:59:06.253697 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:06.254242 8038 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:06.251160 8084 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:06.252988 8045 raft_consensus.cc:2955] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Committing config change with OpId 8.9: config changed from index 8 to 9, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:06.263072 10651 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election won for term 8
W20260616 19:59:06.263553 10788 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260616 19:59:06.255124 10757 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:06.224632 10721 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260616 19:59:06.294719 10757 raft_consensus.cc:697] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:06.295694 10757 consensus_queue.cc:237] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:06.269382 10651 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:06.312213 8038 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:59:06.320407 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 100012us
I20260616 19:59:06.297289 10558 raft_consensus.cc:1027] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: attempt to promote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260616 19:59:06.334017 10651 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:06.336113 7866 catalog_manager.cc:5696] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:59:06.337459 10657 raft_consensus.cc:993] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:06.337944 10657 raft_consensus.cc:1081] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:06.353931 8045 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260616 19:59:06.348634 10791 log.cc:927] Time spent T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.076s user 0.001s sys 0.000s
I20260616 19:59:06.378227 8036 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260616 19:59:06.380771 8117 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260616 19:59:06.381426 8038 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260616 19:59:06.382732 8039 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260616 19:59:06.384740 7963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49190
I20260616 19:59:06.385330 7963 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:06.385669 7963 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:06.390868 10558 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260616 19:59:06.475822 10793 log.cc:927] Time spent T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20260616 19:59:06.382956 10697 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
W20260616 19:59:06.482390 10790 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.099s user 0.000s sys 0.001s
W20260616 19:59:06.555189 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:06.523940 7963 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:06.577445 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 97390.6us
I20260616 19:59:06.601079 7961 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:06.644205 7967 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:59:06.697858 7967 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260616 19:59:06.869616 10803 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.263s user 0.000s sys 0.001s
I20260616 19:59:06.913071 8041 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260616 19:59:06.913899 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:07.116066 8818 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:07.091517 10719 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:07.118757 10719 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:07.121342 10719 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:07.146981 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 78302.9us
I20260616 19:59:07.133842 10558 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:07.165650 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:07.158823 10558 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.190797 8817 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260616 19:59:07.192330 8819 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:59:07.173938 10801 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260616 19:59:07.198348 7968 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:07.202514 10789 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.820s user 0.011s sys 0.000s
I20260616 19:59:07.215991 10727 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:07.216256 10702 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:07.219039 10558 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:07.222905 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:07.227066 8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52470 (request call id 115) took 1129 ms. Trace:
I20260616 19:59:07.227505 8815 rpcz_store.cc:276] 0616 19:59:06.097924 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:06.098448 (+ 524us) service_pool.cc:224] Handling call
0616 19:59:06.138984 (+ 40536us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:06.170755 (+ 31771us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:06.174230 (+ 3475us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:06.200644 (+ 26414us) log.cc:844] Serialized 26 byte log entry
0616 19:59:06.294084 (+ 93440us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:06.297942 (+ 3858us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:06.312661 (+ 14719us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:07.201919 (+889258us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:59:07.226765 (+ 24846us) raft_consensus.cc:1685] finished
0616 19:59:07.226793 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:07.226987 (+ 194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7678,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":14130,"spinlock_wait_cycles":3458432,"thread_start_us":76568,"threads_started":1,"wal-append.queue_time_us":129081}
I20260616 19:59:07.232206 7968 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:07.232656 7969 raft_consensus.cc:606] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Received request to transfer leadership
I20260616 19:59:07.232960 7969 raft_consensus.cc:614] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:07.233314 7966 raft_consensus.cc:606] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:07.233795 7966 raft_consensus.cc:614] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:07.236477 7964 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260616 19:59:07.237991 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 51502us
I20260616 19:59:07.240535 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2306) took 1122 ms. Trace:
I20260616 19:59:07.240935 7968 rpcz_store.cc:276] 0616 19:59:06.113584 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:06.122007 (+ 8423us) service_pool.cc:224] Handling call
0616 19:59:07.197467 (+1075460us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b58005729d0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f24148a 00007fe86f2c3827 00007fe86f2dc3c7 00007fe86f2ddbbe 00007fe86f2e965c 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c
0616 19:59:07.235586 (+ 38119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752857344}
I20260616 19:59:07.244642 10801 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260616 19:59:07.262161 10799 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.413s user 0.002s sys 0.008s
I20260616 19:59:07.290143 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:07.291047 8041 raft_consensus.cc:3055] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:07.291477 8041 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:07.328302 8041 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:07.380980 8041 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:07.381328 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 46752.9us
I20260616 19:59:07.412797 8115 raft_consensus.cc:1275] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260616 19:59:07.381707 10606 consensus_queue.cc:1175] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b022cac8cb5e46b7a39a748e81e257a7" responder_term: 8 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260616 19:59:07.412532 10662 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:07.408488 8118 raft_consensus.cc:2955] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:07.425045 8111 raft_consensus.cc:3055] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:07.433439 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:07.433914 10808 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:07.434458 10808 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.435179 8044 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:59:07.420933 8814 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEARNER]: Advancing to term 8
I20260616 19:59:07.436764 10808 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:07.439333 10558 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:07.446280 10696 consensus_queue.cc:1048] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:07.447151 10757 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:59:07.450943 10739 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:07.451453 10739 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.453552 10739 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:07.423976 10820 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:07.456445 10820 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.456974 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:07.457928 10802 log.cc:927] Time spent T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.617s user 0.004s sys 0.016s
I20260616 19:59:07.434147 10702 raft_consensus.cc:2955] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Committing config change with OpId 9.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:07.425439 8111 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:07.479031 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:07.446688 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49182 (request call id 78) took 1256 ms. Trace:
I20260616 19:59:07.480319 7969 rpcz_store.cc:276] 0616 19:59:06.190674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:06.194241 (+ 3567us) service_pool.cc:224] Handling call
0616 19:59:07.446502 (+1252261us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002fc954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2dfc33 00007fe8742add0a 00007fe86d831934 00007fe86d8318c2 00007fe86d8317f1 00007fe86d83176c 00007fe86d830a22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:07.446603 (+ 101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369178496}
I20260616 19:59:07.465191 8041 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
W20260616 19:59:07.470193 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:07.479139 8111 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.488027 7934 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:07.474812 10718 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:07.489895 10558 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:07.490216 10718 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.492856 8044 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:59:07.495579 8110 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:07.497237 10696 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:07.491753 8816 raft_consensus.cc:2955] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:07.495074 7935 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:07.503223 10558 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:07.511912 7963 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:07.511503 8814 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:59:07.511690 10792 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:07.513067 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41276 (request call id 29) took 1326 ms. Trace:
I20260616 19:59:07.533123 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:49190 (request call id 1679) took 1143 ms. Trace:
W20260616 19:59:07.514145 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:07.532428 7966 rpcz_store.cc:276] 0616 19:59:06.186674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:06.186825 (+ 151us) service_pool.cc:224] Handling call
0616 19:59:07.512916 (+1326091us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002fc954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2dfc33 00007fe8742add0a 00007fe86d831934 00007fe86d8318c2 00007fe86d8317f1 00007fe86d83176c 00007fe86d830a22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:07.513016 (+ 100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364613504}
I20260616 19:59:07.533473 7963 rpcz_store.cc:276] 0616 19:59:06.384453 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:06.384616 (+ 163us) service_pool.cc:224] Handling call
0616 19:59:07.511418 (+1126802us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4c00217480. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86d5af120 00007fe86d5c0129 00007fe86d5c083f 00007fe86d5849ab 00007fe86d5b263c 00007fe86d5b2afe 00007fe86d8409eb 00007fe86f241875 00007fe86f2926ef 00007fe86f2dce35 00007fe8742ad3d0
0616 19:59:07.527721 (+ 16303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35431808}
I20260616 19:59:07.536473 8038 raft_consensus.cc:2955] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:07.550479 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:07.552059 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:07.552659 8118 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:07.517976 10807 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260616 19:59:07.555712 10718 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:07.556077 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:07.537971 10792 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:07.556721 8110 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:07.558413 10792 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:07.558543 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:07.561825 8118 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:07.562449 10626 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:07.562863 10626 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:07.563212 10626 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:07.532130 10696 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260616 19:59:07.570188 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260616 19:59:07.541364 10795 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:07.571695 10795 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:07.573839 10795 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:07.575999 10626 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.578644 10626 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:07.545528 10820 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:07.573869 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 226) took 1689 ms. Trace:
I20260616 19:59:07.584105 7865 rpcz_store.cc:276] 0616 19:59:05.884565 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:05.884737 (+ 172us) service_pool.cc:224] Handling call
0616 19:59:07.573820 (+1689083us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:06.073302 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:06.083536 (+ 10234us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:06.083584 (+ 48us) write_op.cc:435] Acquired schema lock
0616 19:59:06.083604 (+ 20us) tablet.cc:662] Decoding operations
0616 19:59:06.084014 (+ 410us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:06.084077 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:06.084105 (+ 28us) tablet.cc:685] Acquiring locks for 2 operations
0616 19:59:06.084453 (+ 348us) tablet.cc:701] Row locks acquired
0616 19:59:06.084472 (+ 19us) write_op.cc:260] PREPARE: finished
0616 19:59:06.084634 (+ 162us) write_op.cc:270] Start()
0616 19:59:06.084745 (+ 111us) write_op.cc:276] Timestamp: P: 1781639946084583 usec, L: 0
0616 19:59:06.084767 (+ 22us) op_driver.cc:348] REPLICATION: starting
0616 19:59:06.085384 (+ 617us) log.cc:844] Serialized 919 byte log entry
0616 19:59:06.087484 (+ 2100us) op_driver.cc:464] REPLICATION: finished
0616 19:59:06.087886 (+ 402us) write_op.cc:301] APPLY: starting
0616 19:59:06.087960 (+ 74us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:06.088085 (+ 125us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:06.088212 (+ 127us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:06.088554 (+ 342us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:06.088715 (+ 161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0616 19:59:06.088757 (+ 42us) write_op.cc:312] APPLY: finished
0616 19:59:06.088942 (+ 185us) log.cc:844] Serialized 29 byte log entry
0616 19:59:06.089104 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:06.089198 (+ 94us) write_op.cc:454] Released schema lock
0616 19:59:06.089236 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:06.089558 (+ 322us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:06.089585 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5745,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":217,"apply.run_cpu_time_us":1643,"apply.run_wall_time_us":1644,"num_ops":2,"prepare.queue_time_us":220,"prepare.run_cpu_time_us":2780,"prepare.run_wall_time_us":21403,"replication_time_us":2610}]]}
I20260616 19:59:07.590307 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:07.592123 7969 raft_consensus.cc:1240] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:07.593056 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:07.595183 7961 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:07.602279 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:07.605705 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:07.623505 10718 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:07.626137 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:07.628829 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 46317us
I20260616 19:59:07.630246 7970 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:07.549728 10558 raft_consensus.cc:697] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:07.675944 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:07.664556 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:07.669400 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:07.666481 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:07.688316 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:07.707598 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:59:07.708849 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:07.720046 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 98) took 2474 ms. Trace:
I20260616 19:59:07.670862 10718 consensus_queue.cc:1243] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260616 19:59:07.664312 10811 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:07.666818 10662 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:07.673895 10786 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:07.737622 7916 auto_rebalancer.cc:855] Removing replace marker failed: Replica b022cac8cb5e46b7a39a748e81e257a7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
W20260616 19:59:07.749217 7916 auto_rebalancer.cc:863] Could not move replica: Illegal state: not currently leader
W20260616 19:59:07.734426 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 73839us
I20260616 19:59:07.720594 7866 rpcz_store.cc:276] 0616 19:59:05.233304 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:05.236900 (+ 3596us) service_pool.cc:224] Handling call
0616 19:59:07.707446 (+2470546us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:06.346895 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:06.515164 (+168269us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:06.515203 (+ 39us) write_op.cc:435] Acquired schema lock
0616 19:59:06.515231 (+ 28us) tablet.cc:662] Decoding operations
0616 19:59:06.515606 (+ 375us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:06.515665 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:06.515693 (+ 28us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:59:06.515884 (+ 191us) tablet.cc:701] Row locks acquired
0616 19:59:06.515904 (+ 20us) write_op.cc:260] PREPARE: finished
0616 19:59:06.516041 (+ 137us) write_op.cc:270] Start()
0616 19:59:06.516165 (+ 124us) write_op.cc:276] Timestamp: P: 1781639946515999 usec, L: 0
0616 19:59:06.516186 (+ 21us) op_driver.cc:348] REPLICATION: starting
0616 19:59:06.516823 (+ 637us) log.cc:844] Serialized 536 byte log entry
0616 19:59:06.874722 (+357899us) op_driver.cc:464] REPLICATION: finished
0616 19:59:07.185252 (+310530us) write_op.cc:301] APPLY: starting
0616 19:59:07.187225 (+ 1973us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:07.193517 (+ 6292us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:07.193654 (+ 137us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:07.193898 (+ 244us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:07.194075 (+ 177us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:59:07.194133 (+ 58us) write_op.cc:312] APPLY: finished
0616 19:59:07.194341 (+ 208us) log.cc:844] Serialized 23 byte log entry
0616 19:59:07.194531 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:07.194619 (+ 88us) write_op.cc:454] Released schema lock
0616 19:59:07.194660 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:07.195016 (+ 356us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:07.195050 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":603608,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":298175,"apply.run_cpu_time_us":1838,"apply.run_wall_time_us":14459,"num_ops":1,"prepare.queue_time_us":1475,"prepare.run_cpu_time_us":3100,"prepare.run_wall_time_us":229188,"replication_time_us":358378,"thread_start_us":271249,"threads_started":1}]]}
W20260616 19:59:07.743546 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 50739.5us
W20260616 19:59:07.749563 7916 auto_rebalancer.cc:301] scheduled replica move failed to complete: Illegal state: not currently leader
I20260616 19:59:07.724318 10588 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
W20260616 19:59:07.802584 10835 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260616 19:59:07.703912 10600 consensus_queue.cc:1059] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260616 19:59:07.803753 10786 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:07.807049 10828 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.101s user 0.000s sys 0.001s
I20260616 19:59:07.734288 10719 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:07.709668 10664 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
W20260616 19:59:07.805980 10824 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260616 19:59:07.834513 10809 raft_consensus.cc:3055] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:07.733776 10811 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:07.834982 10809 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:07.835887 10809 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.836557 10811 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:07.822084 8114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53906
I20260616 19:59:07.792667 10662 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:07.825918 10664 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:59:07.837098 10809 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:07.817624 10558 consensus_queue.cc:237] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:07.840492 10719 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:07.853746 10724 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:07.862834 10626 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:07.859980 8111 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:07.867627 8038 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:59:07.868449 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:07.869297 8041 raft_consensus.cc:1217] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 3.4->[3.5-3.5] Dedup: 3.5->[]
I20260616 19:59:07.872601 10836 raft_consensus.cc:993] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower de4c5ce3ea8345ae9068f825b8d60e19 to start an election
I20260616 19:59:07.890566 8041 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260616 19:59:07.875936 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:07.892038 8038 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:07.878518 10662 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:07.919999 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:07.920001 10837 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
W20260616 19:59:07.930847 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 238268us
I20260616 19:59:07.906399 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:07.919634 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:08.023684 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 71948.2us
W20260616 19:59:08.028811 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 122545us
W20260616 19:59:08.029651 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 359428us
I20260616 19:59:07.898900 10807 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:07.920934 10836 raft_consensus.cc:1081] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Signalling peer de4c5ce3ea8345ae9068f825b8d60e19 to start an election
W20260616 19:59:08.045426 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 253360us
I20260616 19:59:08.047111 8082 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:07.918716 10786 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:07.890084 10724 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.044739 10662 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:08.055007 10472 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:08.080591 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:08.097060 10811 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:08.101104 10843 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.101655 10843 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:08.103746 10843 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:08.051039 10807 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.127974 8110 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260616 19:59:08.311148 10826 log.cc:927] Time spent T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.153s user 0.000s sys 0.000s
I20260616 19:59:08.346043 8785 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:08.318890 10825 log.cc:927] Time spent T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.135s user 0.000s sys 0.000s
W20260616 19:59:08.357475 10847 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.201s user 0.001s sys 0.000s
I20260616 19:59:08.357944 10806 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:59:08.362360 10850 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.206s user 0.000s sys 0.000s
W20260616 19:59:08.381723 10842 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.323s user 0.001s sys 0.000s
W20260616 19:59:08.384227 10851 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.264s user 0.000s sys 0.002s
I20260616 19:59:08.390360 10858 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.391062 10858 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.393573 10858 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:08.394172 10858 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.385569 10864 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.396130 10864 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:08.397709 10841 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.348s user 0.000s sys 0.001s
I20260616 19:59:08.398844 10864 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:08.394732 10858 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.400665 10807 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:08.403119 10861 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.403429 10724 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:08.403777 10861 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.406364 10856 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.409235 10856 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.416179 10710 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.417003 10710 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:08.437645 10727 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.438427 10727 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.353590 10719 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:59:08.370337 10662 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.444975 10662 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:08.463557 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 385487us
I20260616 19:59:08.447629 10662 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:08.472410 8111 consensus_meta.cc:287] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.583s user 0.007s sys 0.003s
I20260616 19:59:08.472975 8111 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:08.473630 8114 raft_consensus.cc:606] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Received request to transfer leadership
I20260616 19:59:08.474027 8114 raft_consensus.cc:614] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260616 19:59:08.474472 10770 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260616 19:59:08.481142 10839 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.481302 10856 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.482082 10839 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.486617 10727 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.486894 10556 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.487372 10556 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.489351 10556 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:08.494302 10626 consensus_meta.cc:287] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.569s user 0.007s sys 0.003s
W20260616 19:59:08.520310 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 390979us
I20260616 19:59:08.521453 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:08.523893 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.419616 10710 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:08.528560 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.502661 10658 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.406411 10861 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:08.514581 10809 consensus_meta.cc:287] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.637s user 0.011s sys 0.000s
I20260616 19:59:08.558727 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.569427 10858 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:08.572842 8113 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260616 19:59:08.484053 10839 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.572466 10658 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.581287 10658 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:08.582243 8111 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 1988) took 2463 ms (client timeout 1542 ms). Trace:
W20260616 19:59:08.582693 8111 rpcz_store.cc:269] 0616 19:59:06.109494 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:06.109599 (+ 105us) service_pool.cc:224] Handling call
0616 19:59:07.424935 (+1315336us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b540014a620. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f25d1a0 00007fe86f2e91f8 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:07.859422 (+434487us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b58007100d0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f24148a 00007fe86f2c3827 00007fe86f2dc3c7 00007fe86f2ddbbe 00007fe86f2e965c 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c
0616 19:59:08.572263 (+712841us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54002cbd54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:08.572406 (+ 143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479578112}
I20260616 19:59:08.537326 10472 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:08.527429 10612 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:08.585484 10612 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
W20260616 19:59:08.591774 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 70430.9us
I20260616 19:59:08.624233 10702 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:08.626132 10806 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.626859 10806 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:08.629019 10815 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:59:08.632709 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.634946 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.635354 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.636538 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.638342 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.588060 10612 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.655766 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.657042 10612 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.658007 10556 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.658649 10556 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.658769 10612 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:08.661470 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 596858us
I20260616 19:59:08.664070 10806 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.665921 10558 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.666432 10558 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.671054 8785 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:08.675606 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.672755 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:08.676481 7970 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 7.
I20260616 19:59:08.678136 8113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2004) took 1264 ms. Trace:
I20260616 19:59:08.678520 8113 rpcz_store.cc:276] 0616 19:59:07.407275 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:07.407391 (+ 116us) service_pool.cc:224] Handling call
0616 19:59:08.572419 (+1165028us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:08.573356 (+ 937us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:08.671912 (+ 98556us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002cbd50. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:08.672073 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2678613888}
I20260616 19:59:08.682371 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:08.688387 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.689909 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:08.690762 8044 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:08.682389 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:08.693126 7967 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:08.683008 8040 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:08.697026 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:08.664494 10556 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.710762 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:08.713689 10739 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.714300 10739 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.714660 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:08.715287 8786 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:08.675109 10387 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:08.716279 10698 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:08.716827 10698 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:08.679905 10839 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.311640 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260616 19:59:08.776013 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 62948.3us
I20260616 19:59:08.743894 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:09.555119 8037 raft_consensus.cc:2410] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260616 19:59:08.694768 10806 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:09.552435 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:08.718760 10658 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.591275 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:09.591418 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:09.145390 10831 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.144s user 0.001s sys 0.000s
I20260616 19:59:08.781416 10865 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:09.606550 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:08.859614 10739 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.836848 10698 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.844357 10647 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:59:09.469028 10866 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.688s user 0.000s sys 0.001s
I20260616 19:59:09.609382 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:09.572419 10558 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.800815 10612 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:08.819278 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:09.115111 10556 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:08.716318 10387 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.614266 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:09.607784 10626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.113s user 0.011s sys 0.011s
I20260616 19:59:09.154459 10727 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:09.605858 10406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s user 0.011s sys 0.039s
I20260616 19:59:08.789511 10861 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.821s
I20260616 19:59:09.630306 8037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:51870
I20260616 19:59:09.643741 8037 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:09.644254 8037 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:09.595736 10860 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:09.632581 10612 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:09.595773 10658 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.611778 10822 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:09.606782 10865 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.610426 10806 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.585476 10839 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.626610 10647 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.615823 10739 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:09.635783 10626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.141s user 0.011s sys 0.012s
W20260616 19:59:09.645668 10406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s user 0.011s sys 0.039s
I20260616 19:59:09.646900 10727 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.651571 10860 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.668975 8037 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.658648 10865 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:09.675621 8037 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:09.670367 10739 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.674590 10658 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.638971 10556 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.658397 10612 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.665649 10387 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.683806 10658 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:09.684348 10658 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.658463 10822 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.663146 10806 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.680119 10702 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:09.679880 10558 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.683475 10695 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:09.687805 10387 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.699483 10881 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:09.709185 8783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s user 0.001s sys 0.026s
W20260616 19:59:09.715807 8783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s user 0.001s sys 0.026s
I20260616 19:59:09.717178 7933 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:09.705614 10387 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.699903 10558 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.703104 10702 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.718307 10883 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.701387 10695 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.721427 10861 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.722144 10879 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.731305 10880 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.735612 10880 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.735991 10387 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.736532 10387 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.720875 10881 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.736976 10387 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.737697 10880 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.738206 10880 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.738497 10881 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:09.738624 10880 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.739022 10387 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:09.732622 10727 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.739054 10881 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.747470 10839 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.769343 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:09.796566 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 200344us
I20260616 19:59:09.753924 10612 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:09.755384 10739 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.731408 10883 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.756340 10727 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:09.757863 10860 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:09.758876 10611 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.828828 8011 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:09.746057 10861 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.739910 10879 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.798781 10556 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:09.802613 10839 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.807968 10658 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:09.759124 10881 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.830593 10387 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.833573 10822 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:09.830549 10739 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.834362 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:09.843379 10806 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.830336 10612 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:09.856307 10876 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.147s user 0.000s sys 0.001s
W20260616 19:59:09.880472 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 82851.5us
I20260616 19:59:09.882045 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:09.882794 8112 raft_consensus.cc:606] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:09.883421 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:09.885026 8114 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:59:09.885553 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:09.888944 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:09.890035 10900 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:09.890605 10900 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.845613 10611 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.876507 10658 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.860379 10839 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:09.917222 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 82059.4us
I20260616 19:59:09.880944 10806 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:09.891379 10558 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.925452 10611 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:09.924786 10885 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.926268 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:09.926244 10885 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.927009 8112 raft_consensus.cc:606] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:09.928293 10886 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.928723 10885 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.928871 10886 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:09.931241 10886 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.932838 8112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:09.874238 10387 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.933432 8112 raft_consensus.cc:606] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:09.900384 10898 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.925510 10893 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:09.937515 10893 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.937901 10893 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:09.940344 8009 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:09.878448 10739 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.945199 8039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d38debbe377040e48b9e248db8babc55"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:59:09.945794 8039 raft_consensus.cc:606] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:09.887070 10612 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.922451 10702 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.922521 10698 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:09.933140 10700 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:09.948263 10880 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.953548 10698 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.960440 10894 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:09.960546 10727 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:09.961138 10894 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.961207 10727 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.962268 10883 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.951793 10695 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.943555 10898 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.964129 10879 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.964641 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 254) took 1291 ms. Trace:
I20260616 19:59:09.965027 7968 rpcz_store.cc:276] 0616 19:59:08.665799 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.668622 (+ 2823us) service_pool.cc:224] Handling call
0616 19:59:09.956347 (+1287725us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6a3e 000000000036f859 00007fe8742348e0 00007fe8742fd7c4 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:09.956470 (+ 123us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:09.956596 (+ 126us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:09.956610 (+ 14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:09.956646 (+ 36us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:09.956664 (+ 18us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:09.956696 (+ 32us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:09.956770 (+ 74us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5981696}
I20260616 19:59:09.966696 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:09.952826 8815 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:09.967280 7968 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:09.967772 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1718) took 1210 ms. Trace:
I20260616 19:59:09.968549 10865 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.971588 10865 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.955554 10556 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:09.972496 8812 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:09.973963 10893 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:09.921897 10658 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.975584 10893 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.975672 10900 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:09.968052 7968 rpcz_store.cc:276] 0616 19:59:08.757706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.966439 (+1208733us) service_pool.cc:224] Handling call
0616 19:59:09.967716 (+ 1277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:09.976925 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:09.937003 10806 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:09.977567 7968 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:09.977968 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2356) took 1235 ms. Trace:
I20260616 19:59:09.978258 7968 rpcz_store.cc:276] 0616 19:59:08.742661 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.976713 (+1234052us) service_pool.cc:224] Handling call
0616 19:59:09.977928 (+ 1215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:09.978906 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:09.979476 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2357) took 1236 ms. Trace:
I20260616 19:59:09.979768 7968 rpcz_store.cc:276] 0616 19:59:08.742956 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.978737 (+1235781us) service_pool.cc:224] Handling call
0616 19:59:09.979439 (+ 702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:09.980751 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:09.981437 7968 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:09.982016 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1715) took 1264 ms. Trace:
I20260616 19:59:09.982396 7968 rpcz_store.cc:276] 0616 19:59:08.717793 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.980476 (+1262683us) service_pool.cc:224] Handling call
0616 19:59:09.981954 (+ 1478us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:10.086830 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 145135us
I20260616 19:59:10.095456 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:10.097482 10387 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.101688 10612 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.101852 10860 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.102442 10860 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.103874 10806 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:10.097608 10905 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.168s user 0.001s sys 0.000s
I20260616 19:59:10.118319 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.118687 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:10.118896 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.119295 8110 raft_consensus.cc:3055] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:10.119594 8110 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:09.988773 10889 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.120591 8110 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.123111 8110 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:10.087787 10865 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.095991 10879 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:09.956251 10700 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.126806 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.129354 10861 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.129274 10865 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.115391 10558 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.124974 10904 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.127455 10889 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.130767 10895 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.124526 10626 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.131860 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1709) took 1448 ms. Trace:
I20260616 19:59:10.152503 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.133846 8113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:10.155926 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.208272 8113 raft_consensus.cc:606] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:10.155459 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.156379 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.120133 10902 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:10.138767 8110 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:10.154506 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.210705 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.213186 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.211771 10891 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:10.216835 8037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2343) took 1379 ms. Trace:
I20260616 19:59:10.206555 7961 rpcz_store.cc:276] 0616 19:59:08.682925 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.683140 (+ 215us) service_pool.cc:224] Handling call
0616 19:59:10.131055 (+1447915us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.131275 (+ 220us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.131475 (+ 200us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.131507 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.131558 (+ 51us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:10.131593 (+ 35us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.131655 (+ 62us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.131777 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905987840}
I20260616 19:59:10.241742 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.255357 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260616 19:59:10.256506 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1700) took 1576 ms. Trace:
I20260616 19:59:10.257232 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260616 19:59:10.210918 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 78141.5us
I20260616 19:59:10.258323 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:10.135542 10860 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.247471 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.218297 8113 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:10.248486 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:10.249159 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:10.250515 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.250641 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1701) took 1569 ms. Trace:
I20260616 19:59:10.266629 7963 rpcz_store.cc:276] 0616 19:59:08.680737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.683964 (+ 3227us) service_pool.cc:224] Handling call
0616 19:59:10.249687 (+1565723us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.249942 (+ 255us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.250185 (+ 243us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.250223 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.250305 (+ 82us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:10.250351 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.250426 (+ 75us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.250571 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963227904}
I20260616 19:59:10.253890 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.252313 8037 rpcz_store.cc:276] 0616 19:59:08.837649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.687834 (+850185us) service_pool.cc:224] Handling call
0616 19:59:09.688254 (+ 420us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:09.835323 (+147069us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:09.835352 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:09.835820 (+ 468us) log.cc:844] Serialized 26 byte log entry
0616 19:59:09.944412 (+108592us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:09.944668 (+ 256us) log.cc:844] Serialized 14 byte log entry
0616 19:59:10.216406 (+271738us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.216508 (+ 102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:10.216572 (+ 64us) raft_consensus.cc:1685] finished
0616 19:59:10.216596 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.216764 (+ 168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":580,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":70165,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":7117,"thread_start_us":525719,"threads_started":3,"wal-append.queue_time_us":93339}
I20260616 19:59:10.256913 7970 rpcz_store.cc:276] 0616 19:59:08.680436 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.680557 (+ 121us) service_pool.cc:224] Handling call
0616 19:59:10.255767 (+1575210us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.255954 (+ 187us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.256192 (+ 238us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.256218 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.256261 (+ 43us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:10.256293 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.256349 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.256451 (+ 102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933597952}
I20260616 19:59:10.132707 10907 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.260106 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.248090 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.205857 10611 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.268476 7961 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:10.279105 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1717) took 1535 ms. Trace:
I20260616 19:59:10.265908 7969 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
W20260616 19:59:10.277154 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 299634us
I20260616 19:59:10.304231 8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52458 (request call id 318) took 1521 ms. Trace:
I20260616 19:59:10.279436 7961 rpcz_store.cc:276] 0616 19:59:08.743690 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.253672 (+1509982us) service_pool.cc:224] Handling call
0616 19:59:10.279041 (+ 25369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.306115 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.212425 10892 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.277745 8115 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:10.205561 10909 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:10.208289 10658 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:10.304592 8820 rpcz_store.cc:276] 0616 19:59:08.758142 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.789463 (+ 31321us) service_pool.cc:224] Handling call
0616 19:59:10.248956 (+1459493us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b5800003998. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:10.279411 (+ 30455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68084480}
I20260616 19:59:10.301999 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:10.307178 7969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1707) took 1624 ms (client timeout 1698 ms). Trace:
I20260616 19:59:10.254026 10806 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.254568 10861 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.154608 10879 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.320024 8009 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:10.309602 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:10.317198 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 145598us
I20260616 19:59:10.270617 10612 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:10.319175 7969 rpcz_store.cc:269] 0616 19:59:08.682384 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.683378 (+ 994us) service_pool.cc:224] Handling call
0616 19:59:10.250293 (+1566915us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:10.307120 (+ 56827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1965148416}
I20260616 19:59:10.319731 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.211226 10558 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.281093 10893 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.304075 10611 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.323524 10897 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.314122 10839 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.325209 10700 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.320236 10892 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.240607 10626 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.332810 7970 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:10.320663 10811 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.306241 10860 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.338714 8009 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:10.337857 8085 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:10.341135 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:10.318243 10868 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.211783 10895 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.318325 10880 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.334848 7963 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:10.340322 10387 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:10.376508 10387 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:10.376339 10811 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.377451 10387 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.378451 10811 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.378652 10882 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.379031 10880 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.364567 7969 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:10.332041 10806 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.345443 10900 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.369369 10697 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:10.369715 10558 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.370522 10626 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.370908 10839 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260616 19:59:10.366245 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 107721us
I20260616 19:59:10.371331 10868 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.401432 10919 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.374169 7961 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:10.372910 10860 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.375684 10895 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.380195 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.236505 10904 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.379204 10882 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.363751 10888 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.402006 10919 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.380121 10897 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.384733 10889 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.474507 10916 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.334786 10861 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.349355 10658 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.464566 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1704) took 1783 ms. Trace:
I20260616 19:59:10.386322 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.414847 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1710) took 1731 ms. Trace:
I20260616 19:59:10.484932 7970 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:10.469432 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.486814 7964 raft_consensus.cc:2410] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260616 19:59:10.487211 7964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1716) took 1749 ms (client timeout 1580 ms). Trace:
I20260616 19:59:10.487605 10861 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:10.487540 7964 rpcz_store.cc:269] 0616 19:59:08.737385 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.747189 (+ 9804us) service_pool.cc:224] Handling call
0616 19:59:10.469291 (+1722102us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:10.487146 (+ 17855us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11123456}
I20260616 19:59:10.488447 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:10.488664 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.488826 8082 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:10.489022 8820 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:10.489768 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.301483 10891 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.490283 10916 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.491225 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.485944 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:10.406986 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:10.406443 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 142550us
I20260616 19:59:10.484519 7966 rpcz_store.cc:276] 0616 19:59:08.681536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.683293 (+ 1757us) service_pool.cc:224] Handling call
0616 19:59:10.463711 (+1780418us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.463945 (+ 234us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.464180 (+ 235us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.464215 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.464262 (+ 47us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:10.464296 (+ 34us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.464358 (+ 62us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.464498 (+ 140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60933376}
I20260616 19:59:10.492309 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.494889 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.485196 7965 rpcz_store.cc:276] 0616 19:59:08.683214 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.683324 (+ 110us) service_pool.cc:224] Handling call
0616 19:59:10.414057 (+1730733us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.414288 (+ 231us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.414481 (+ 193us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.414510 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.414549 (+ 39us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:10.414580 (+ 31us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.414640 (+ 60us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.414772 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1992067328}
I20260616 19:59:10.496409 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:10.497053 8041 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:10.495419 7966 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 5.
I20260616 19:59:10.389808 10697 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.303081 10907 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.320924 10909 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.401034 10839 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:10.513500 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:10.514510 8045 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:10.340574 10612 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:10.401331 7916 auto_rebalancer.cc:855] Removing replace marker failed: Replica 6827184e3cef42c2ada15b2d31afee4b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
W20260616 19:59:10.527974 7916 auto_rebalancer.cc:863] Could not move replica: Illegal state: not currently leader
W20260616 19:59:10.528303 7916 auto_rebalancer.cc:301] scheduled replica move failed to complete: Illegal state: not currently leader
I20260616 19:59:10.529237 10612 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.402558 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:10.382714 10811 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.533727 10811 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.535048 10909 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:10.500049 10889 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:10.349133 10893 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.381160 10892 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:10.531635 8785 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:10.405575 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:10.484437 10658 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.414513 8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52458 (request call id 314) took 1702 ms. Trace:
I20260616 19:59:10.496745 10878 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.438079 10822 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.412259 7969 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:10.497835 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:10.546177 8820 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:10.496778 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.386400 8085 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:10.331663 10902 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.503587 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.468832 10865 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.512370 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.512662 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1712) took 1828 ms. Trace:
I20260616 19:59:10.513435 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:10.397341 10885 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:10.492266 10920 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.516707 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:10.557164 8041 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:10.514612 10914 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260616 19:59:10.520975 10806 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.521951 10860 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.482733 10880 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:10.522181 10904 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.502955 10697 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:10.523572 8814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52474 (request call id 192) took 1806 ms. Trace:
I20260616 19:59:10.525872 10882 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.525758 10907 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.527472 10879 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.465917 10739 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.537906 8819 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:10.544839 10695 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:10.541079 10893 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.438292 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1711) took 1754 ms. Trace:
I20260616 19:59:10.542999 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:10.541522 8817 rpcz_store.cc:276] 0616 19:59:08.720203 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.733062 (+ 12859us) service_pool.cc:224] Handling call
0616 19:59:08.733452 (+ 390us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:09.599089 (+865637us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:09.599135 (+ 46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:09.599191 (+ 56us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:09.599234 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.414173 (+814939us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54004b78d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:10.414242 (+ 69us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.414433 (+ 191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990266112}
I20260616 19:59:10.544912 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:10.550305 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1713) took 1857 ms. Trace:
I20260616 19:59:10.547544 7933 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:10.571599 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2352) took 1853 ms. Trace:
I20260616 19:59:10.614246 8045 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 7.
I20260616 19:59:10.552744 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:10.553470 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 150977us
I20260616 19:59:10.615427 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:10.547353 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 161047us
I20260616 19:59:10.612716 7967 rpcz_store.cc:276] 0616 19:59:08.683477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.693902 (+ 10425us) service_pool.cc:224] Handling call
0616 19:59:10.437470 (+1743568us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.437709 (+ 239us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.437925 (+ 216us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.437961 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.438007 (+ 46us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:10.438035 (+ 28us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.438094 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.438212 (+ 118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1955375488}
I20260616 19:59:10.614142 7969 rpcz_store.cc:276] 0616 19:59:08.692907 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.548769 (+1855862us) service_pool.cc:224] Handling call
0616 19:59:10.549353 (+ 584us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.549713 (+ 360us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.549779 (+ 66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.549871 (+ 92us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:10.549949 (+ 78us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.550060 (+ 111us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.550214 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.400520 10900 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.556886 7966 rpcz_store.cc:276] 0616 19:59:08.683714 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.511478 (+1827764us) service_pool.cc:224] Handling call
0616 19:59:10.511891 (+ 413us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.512132 (+ 241us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.512188 (+ 56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.512253 (+ 65us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:10.512300 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.512377 (+ 77us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.512548 (+ 171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.615433 8785 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:10.624269 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.626066 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.559931 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:10.536629 10868 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.535868 10811 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.502887 10888 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:10.629848 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:10.610795 8819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52458 (request call id 315) took 1890 ms (client timeout 1550 ms). Trace:
W20260616 19:59:10.611471 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 122729us
I20260616 19:59:10.516041 10839 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:10.617957 7969 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
W20260616 19:59:10.618643 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 71164.5us
I20260616 19:59:10.614584 7965 rpcz_store.cc:276] 0616 19:59:08.718392 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.570741 (+1852349us) service_pool.cc:224] Handling call
0616 19:59:10.571096 (+ 355us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.571271 (+ 175us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.571297 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.571342 (+ 45us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:10.571378 (+ 36us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.571443 (+ 65us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.571541 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.634182 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2354) took 1912 ms. Trace:
W20260616 19:59:10.610656 10873 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.928s user 0.001s sys 0.004s
I20260616 19:59:10.634558 7969 rpcz_store.cc:276] 0616 19:59:08.722172 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.617538 (+1895366us) service_pool.cc:224] Handling call
0616 19:59:10.617801 (+ 263us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.633934 (+ 16133us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.634121 (+ 187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.636102 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2355) took 1901 ms. Trace:
I20260616 19:59:10.636456 7969 rpcz_store.cc:276] 0616 19:59:08.734876 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.635395 (+1900519us) service_pool.cc:224] Handling call
0616 19:59:10.635691 (+ 296us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.635835 (+ 144us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.635856 (+ 21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.635891 (+ 35us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:10.635919 (+ 28us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.635967 (+ 48us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.636055 (+ 88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.637154 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.618656 7935 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:10.605605 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.617575 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2353) took 1896 ms. Trace:
I20260616 19:59:10.552318 10892 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:10.622541 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.638746 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:10.638980 7968 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:10.638701 7967 rpcz_store.cc:276] 0616 19:59:08.721063 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.616816 (+1895753us) service_pool.cc:224] Handling call
0616 19:59:10.617110 (+ 294us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.617278 (+ 168us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.617302 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.617344 (+ 42us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:10.617373 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.617424 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.617529 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.564546 10861 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:10.640990 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.642369 8118 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:10.638374 7962 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
W20260616 19:59:10.643040 7962 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1703) took 1961 ms (client timeout 2169 ms). Trace:
I20260616 19:59:10.640868 8085 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:10.641505 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 1891) took 1944 ms. Trace:
W20260616 19:59:10.630791 8819 rpcz_store.cc:269] 0616 19:59:08.720522 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.720678 (+ 156us) service_pool.cc:224] Handling call
0616 19:59:10.405411 (+1684733us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b5800003998. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:10.610728 (+205317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15817088}
I20260616 19:59:10.640606 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2360) took 1858 ms. Trace:
I20260616 19:59:10.643265 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.645337 8118 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 2.
I20260616 19:59:10.644919 7967 rpcz_store.cc:276] 0616 19:59:08.781771 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.639843 (+1858072us) service_pool.cc:224] Handling call
0616 19:59:10.640132 (+ 289us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.640318 (+ 186us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.640342 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.640379 (+ 37us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:10.640408 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.640459 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.640549 (+ 90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31744}
I20260616 19:59:10.638305 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2358) took 1895 ms. Trace:
I20260616 19:59:10.646248 7969 rpcz_store.cc:276] 0616 19:59:08.743245 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.637313 (+1894068us) service_pool.cc:224] Handling call
0616 19:59:10.637575 (+ 262us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.637975 (+ 400us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.638004 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.638046 (+ 42us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:10.638083 (+ 37us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.638149 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.638258 (+ 109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.642035 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.640606 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2359) took 1869 ms. Trace:
I20260616 19:59:10.647317 8114 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:10.647377 7968 rpcz_store.cc:276] 0616 19:59:08.771628 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.639800 (+1868172us) service_pool.cc:224] Handling call
0616 19:59:10.640132 (+ 332us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.640318 (+ 186us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.640342 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.640379 (+ 37us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:10.640408 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.640459 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.640549 (+ 90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38272}
W20260616 19:59:10.643425 7962 rpcz_store.cc:269] 0616 19:59:08.681272 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.681964 (+ 692us) service_pool.cc:224] Handling call
0616 19:59:10.605433 (+1923469us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:10.642990 (+ 37557us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28446976}
I20260616 19:59:10.643846 8043 rpcz_store.cc:276] 0616 19:59:08.684898 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.687878 (+ 2980us) service_pool.cc:224] Handling call
0616 19:59:08.719960 (+ 32082us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:09.663792 (+943832us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:09.663820 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:09.663859 (+ 39us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:09.663886 (+ 27us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.629043 (+965157us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b54003a9654. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:10.629108 (+ 65us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.629234 (+ 126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170184576}
I20260616 19:59:10.552675 10822 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.571296 10919 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.609295 10897 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.653067 8786 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:10.616431 10695 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.694152 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.718699 8118 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:10.693717 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:10.713985 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2362) took 1128 ms. Trace:
I20260616 19:59:10.719607 7965 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:59:10.625207 10907 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:10.672794 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2361) took 1099 ms. Trace:
W20260616 19:59:10.725230 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 84436.3us
I20260616 19:59:10.726500 8085 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:10.719854 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 66873.5us
I20260616 19:59:10.655207 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.572189 10885 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.723934 7966 rpcz_store.cc:276] 0616 19:59:09.573199 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.671651 (+1098452us) service_pool.cc:224] Handling call
0616 19:59:10.672128 (+ 477us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.672363 (+ 235us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.672400 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.672456 (+ 56us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:59:10.672500 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.672570 (+ 70us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.672715 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.719570 7969 rpcz_store.cc:276] 0616 19:59:09.586082 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.663803 (+1077721us) service_pool.cc:224] Handling call
0616 19:59:10.713291 (+ 49488us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.713532 (+ 241us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.713580 (+ 48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.713639 (+ 59us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:10.713685 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.713759 (+ 74us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.713905 (+ 146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:10.659054 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.722540 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:10.642207 10871 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:10.741375 8115 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:10.673316 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.745443 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.746029 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.641844 10861 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.641680 10892 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.629853 10739 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:10.642416 10882 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:10.694053 10879 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:10.748080 8114 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 5.
I20260616 19:59:10.548657 10878 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:10.693331 10558 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260616 19:59:10.716048 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 97421.2us
I20260616 19:59:10.751641 10892 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:10.863483 10861 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:10.864667 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.753069 8085 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:10.556860 10902 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.754381 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.722038 10695 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:10.642097 10916 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:10.967063 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.970064 10698 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.944928 10879 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:10.733985 10868 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:10.972764 10868 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.973317 10868 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:10.867892 10923 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:10.978793 10923 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:10.979300 10923 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:10.724423 10806 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:10.569456 10920 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:10.634675 10898 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:10.658643 10860 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:10.611467 8814 rpcz_store.cc:276] 0616 19:59:08.716872 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.721427 (+ 4555us) service_pool.cc:224] Handling call
0616 19:59:10.438516 (+1717089us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b5800003998. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.473687 (+ 35171us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.502740 (+ 29053us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.505168 (+ 2428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.515453 (+ 10285us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:10.523218 (+ 7765us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.523321 (+ 103us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.523491 (+ 170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4904832}
I20260616 19:59:10.954660 10658 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:10.852499 8011 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 2.190s (SENT) [suppressed 8 similar messages]
W20260616 19:59:11.045495 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 193055us
I20260616 19:59:10.972289 10879 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:10.947515 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.030087 10889 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:11.029875 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:11.030465 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 277452us
I20260616 19:59:11.030520 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.052430 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.052881 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.053074 8115 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:11.031587 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.944871 7866 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 3 to 4, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129), now has a pending config: VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129), VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130), VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131), VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:11.054203 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.033665 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.864497 10739 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:11.052914 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.944693 8117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2039) took 1060 ms. Trace:
I20260616 19:59:10.747499 10871 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:11.056788 8117 rpcz_store.cc:276] 0616 19:59:09.884117 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.884243 (+ 126us) service_pool.cc:224] Handling call
0616 19:59:09.884492 (+ 249us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:09.923868 (+ 39376us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:09.923910 (+ 42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:09.923974 (+ 64us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:09.924017 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:10.944457 (+1020440us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b5400215554. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:10.944504 (+ 47us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:10.944633 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90092928}
I20260616 19:59:10.942842 10558 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:11.059494 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:11.043571 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 106047us
I20260616 19:59:11.060344 8113 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:11.053938 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.063396 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.058447 7866 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 8 to 9, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:11.063872 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.064843 7866 catalog_manager.cc:5696] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 6 to 7, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:11.068655 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.938700 10919 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260616 19:59:11.073172 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.051759 10889 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:11.077327 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:10.938371 10882 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.061530 10871 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:11.059362 10739 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:11.060035 10658 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:11.062325 10558 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:11.043725 10860 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.095465 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.097121 10822 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:11.097280 10857 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:11.084438 10889 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:11.124737 8085 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:11.126180 10885 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:11.127254 10839 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.163614 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.092708 10888 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:11.168104 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.172868 7969 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:11.172868 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:11.168471 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 43851.9us
I20260616 19:59:11.175669 10900 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.176080 10724 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:59:11.176273 8784 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:11.176578 10724 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:11.176738 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.177101 10724 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:11.175805 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.178591 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:11.177394 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 174749us
I20260616 19:59:11.183104 7963 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:11.199242 8786 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:11.200032 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:11.201328 10922 log.cc:927] Time spent T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.119s user 0.000s sys 0.000s
I20260616 19:59:11.201504 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:11.172463 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 111316us
I20260616 19:59:11.225356 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.226294 8037 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:11.236892 8044 raft_consensus.cc:1240] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:11.110350 10857 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.103950 10658 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.261518 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:11.263762 10925 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:11.264439 10925 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:11.281586 7965 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:11.287741 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.289423 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.290839 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:11.294670 7961 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2368) took 1652 ms (client timeout 1973 ms). Trace:
W20260616 19:59:11.295210 7961 rpcz_store.cc:269] 0616 19:59:09.642559 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.305800 (+663241us) service_pool.cc:224] Handling call
0616 19:59:11.294467 (+988667us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b54002085d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:11.294599 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154331008}
I20260616 19:59:11.220329 8784 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:11.243988 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.349615 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2369) took 1699 ms. Trace:
I20260616 19:59:11.259228 10916 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:59:11.365516 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 327597us
I20260616 19:59:11.298501 8812 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52458 (request call id 320) took 2478 ms. Trace:
I20260616 19:59:11.271463 10878 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.206422 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.390369 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.205842 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:11.391707 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 171528us
I20260616 19:59:11.393018 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 258) took 1278 ms. Trace:
I20260616 19:59:11.405963 7969 rpcz_store.cc:276] 0616 19:59:10.114766 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.390748 (+1275982us) service_pool.cc:224] Handling call
0616 19:59:11.391030 (+ 282us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.392446 (+ 1416us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.392502 (+ 56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.392568 (+ 66us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:11.392620 (+ 52us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.392713 (+ 93us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.392952 (+ 239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:11.407824 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 345978us
W20260616 19:59:11.397774 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:11.409873 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:11.350065 7962 rpcz_store.cc:276] 0616 19:59:09.650703 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.700966 (+1050263us) service_pool.cc:224] Handling call
0616 19:59:10.726087 (+ 25121us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.726279 (+ 192us) raft_consensus.cc:1534] Early marking committed up to index 8
0616 19:59:11.046088 (+319809us) log.cc:844] Serialized 14 byte log entry
0616 19:59:11.349268 (+303180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.349322 (+ 54us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:11.349371 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.349437 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.349554 (+ 117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":540428,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":58,"raft.queue_time_us":100,"thread_start_us":622186,"threads_started":2}
W20260616 19:59:11.411250 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 345275us
I20260616 19:59:11.411273 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 259) took 1287 ms. Trace:
I20260616 19:59:11.413169 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.413848 8110 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:59:11.415078 7969 rpcz_store.cc:276] 0616 19:59:10.123951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.407392 (+1283441us) service_pool.cc:224] Handling call
0616 19:59:11.407798 (+ 406us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.408034 (+ 236us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.408074 (+ 40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.408130 (+ 56us) raft_consensus.cc:1645] Marking committed up to 10
0616 19:59:11.410996 (+ 2866us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.411067 (+ 71us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.411212 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:11.416996 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 260) took 1290 ms. Trace:
I20260616 19:59:11.417421 7969 rpcz_store.cc:276] 0616 19:59:10.126364 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.416071 (+1289707us) service_pool.cc:224] Handling call
0616 19:59:11.416385 (+ 314us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.416569 (+ 184us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.416606 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.416664 (+ 58us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:11.416707 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.416793 (+ 86us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.416929 (+ 136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:11.423825 8009 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:11.424211 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.424904 8009 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:11.425690 8009 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:11.425051 8044 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:59:11.427562 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.428172 8043 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:11.401103 10878 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:11.433316 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2367) took 1792 ms. Trace:
I20260616 19:59:11.265561 10925 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.435683 7968 rpcz_store.cc:276] 0616 19:59:09.640574 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.693702 (+1053128us) service_pool.cc:224] Handling call
0616 19:59:11.432503 (+738801us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:11.432699 (+ 196us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.432904 (+ 205us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.432943 (+ 39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.432999 (+ 56us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:11.433042 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.433112 (+ 70us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.433252 (+ 140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2855808}
I20260616 19:59:11.442219 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2374) took 1108 ms. Trace:
I20260616 19:59:11.442751 7968 rpcz_store.cc:276] 0616 19:59:10.334450 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.439923 (+1105473us) service_pool.cc:224] Handling call
0616 19:59:11.440529 (+ 606us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.440738 (+ 209us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.440774 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.440834 (+ 60us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:11.440878 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.440949 (+ 71us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.441077 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:11.437723 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.447139 8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52474 (request call id 197) took 1546 ms. Trace:
I20260616 19:59:11.447721 8815 rpcz_store.cc:276] 0616 19:59:09.900451 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.904126 (+ 3675us) service_pool.cc:224] Handling call
0616 19:59:09.941488 (+ 37362us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:09.950138 (+ 8650us) raft_consensus.cc:1534] Early marking committed up to index 6
0616 19:59:10.469940 (+519802us) log.cc:844] Serialized 14 byte log entry
0616 19:59:11.061079 (+591139us) log.cc:844] Serialized 14 byte log entry
0616 19:59:11.446800 (+385721us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.446853 (+ 53us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:11.446902 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.446972 (+ 70us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.447086 (+ 114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":574718,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":17056,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7093,"spinlock_wait_cycles":22060928,"thread_start_us":867504,"threads_started":3,"wal-append.queue_time_us":274484}
I20260616 19:59:11.450470 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2371) took 1783 ms. Trace:
I20260616 19:59:11.452389 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2372) took 1121 ms. Trace:
I20260616 19:59:11.388108 10916 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.452741 7962 rpcz_store.cc:276] 0616 19:59:10.330723 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.429188 (+1098465us) service_pool.cc:224] Handling call
0616 19:59:11.451702 (+ 22514us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.451866 (+ 164us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.451890 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.451930 (+ 40us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:11.451963 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.452024 (+ 61us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.452141 (+ 117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:11.410656 10891 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:11.444996 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2376) took 1077 ms. Trace:
I20260616 19:59:11.479208 7968 rpcz_store.cc:276] 0616 19:59:10.367721 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.444023 (+1076302us) service_pool.cc:224] Handling call
0616 19:59:11.444403 (+ 380us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.444604 (+ 201us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.444643 (+ 39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.444697 (+ 54us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:11.444741 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.444809 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.444937 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:11.481778 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.429322 10897 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:11.481993 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.426900 10893 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:11.484889 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 57096.5us
I20260616 19:59:11.481877 8082 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:11.428992 10839 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.493379 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.495010 8009 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:11.450901 7961 rpcz_store.cc:276] 0616 19:59:09.667066 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.296384 (+1629318us) service_pool.cc:224] Handling call
0616 19:59:11.296746 (+ 362us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.297033 (+ 287us) raft_consensus.cc:1534] Early marking committed up to index 4
0616 19:59:11.298161 (+ 1128us) log.cc:844] Serialized 14 byte log entry
0616 19:59:11.450137 (+151976us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.450186 (+ 49us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:11.450258 (+ 72us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.450322 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.450420 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":931,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":356,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":123,"spinlock_wait_cycles":29440,"thread_start_us":151374,"threads_started":1,"wal-append.queue_time_us":151433}
W20260616 19:59:11.455963 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 357883us
I20260616 19:59:11.431107 10887 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260616 19:59:11.497985 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 88078.7us
I20260616 19:59:11.498210 8082 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:11.453815 10658 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.451901 10865 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:11.517712 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.519446 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2377) took 1133 ms. Trace:
I20260616 19:59:11.520001 7962 rpcz_store.cc:276] 0616 19:59:10.385917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.497343 (+1111426us) service_pool.cc:224] Handling call
0616 19:59:11.518722 (+ 21379us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.518954 (+ 232us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.518995 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.519054 (+ 59us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:11.519101 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.519204 (+ 103us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.519365 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:11.509775 10806 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:11.522400 10860 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.518574 8043 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:11.491878 10897 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:11.485823 10940 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:11.491621 10878 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:11.389042 8812 rpcz_store.cc:276] 0616 19:59:08.819745 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.843645 (+ 23900us) service_pool.cc:224] Handling call
0616 19:59:09.628453 (+784808us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:09.972960 (+344507us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.298117 (+1325157us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400066154. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:11.298402 (+ 285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790645504}
I20260616 19:59:11.506634 10881 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.498133 10887 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:11.512981 10891 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:11.499519 10757 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:11.541136 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.575068 8038 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:11.543581 10916 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:11.548732 10882 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:11.549407 10935 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:11.513319 10902 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.513229 10888 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260616 19:59:11.587909 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 107982us
I20260616 19:59:11.524904 10658 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:11.577178 10806 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.601635 10658 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:11.602080 10658 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:11.530058 10727 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.563814 10739 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:11.599977 10939 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.191s user 0.002s sys 0.001s
I20260616 19:59:11.566886 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:11.568565 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1714) took 2851 ms. Trace:
I20260616 19:59:11.569317 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:11.562153 7934 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:11.573494 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 74734.9us
I20260616 19:59:11.570326 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.533923 10897 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:11.578217 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.628744 8043 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:11.503916 10893 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.579071 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.631635 7967 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:11.632078 8038 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
W20260616 19:59:11.569979 10868 consensus_meta.cc:287] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.596s user 0.005s sys 0.007s
I20260616 19:59:11.534327 10878 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:11.586841 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 177403us
I20260616 19:59:11.588316 10935 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:11.593331 10406 raft_consensus.cc:993] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
W20260616 19:59:11.594542 10938 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.149s user 0.000s sys 0.000s
W20260616 19:59:11.596992 10931 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.140s user 0.000s sys 0.000s
I20260616 19:59:11.570874 10891 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:11.645354 10891 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:11.645781 8813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52458 (request call id 319) took 2861 ms. Trace:
I20260616 19:59:11.646229 8813 rpcz_store.cc:276] 0616 19:59:08.784036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.796301 (+ 12265us) service_pool.cc:224] Handling call
0616 19:59:10.974789 (+2178488us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5800003998. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:10.975085 (+ 296us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:10.975569 (+ 484us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.975610 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:11.041799 (+ 66189us) log.cc:844] Serialized 26 byte log entry
0616 19:59:11.454357 (+412558us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:11.504890 (+ 50533us) log.cc:844] Serialized 14 byte log entry
0616 19:59:11.631080 (+126190us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.631166 (+ 86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:11.645442 (+ 14276us) raft_consensus.cc:1685] finished
0616 19:59:11.645548 (+ 106us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.645703 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6837,"prepare.queue_time_us":4048,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":7555,"raft.queue_time_us":1888,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":55344384,"thread_start_us":476078,"threads_started":2,"wal-append.queue_time_us":400919}
I20260616 19:59:11.571002 10887 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:11.653136 10811 raft_consensus.cc:1064] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:59:11.582068 10757 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.655117 10757 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:11.552748 10940 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.657248 10940 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.657472 10739 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:11.606745 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.659001 8045 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:11.608302 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.610633 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.661501 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.662132 7968 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
W20260616 19:59:11.566319 10892 consensus_meta.cc:287] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.615s user 0.006s sys 0.015s
W20260616 19:59:11.611812 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 116891us
I20260616 19:59:11.663165 10892 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.664006 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:11.665346 10892 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:11.665488 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:11.666168 8812 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:11.617455 7964 rpcz_store.cc:276] 0616 19:59:08.717505 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.554582 (+1837077us) service_pool.cc:224] Handling call
0616 19:59:11.298709 (+744127us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.298960 (+ 251us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.299008 (+ 48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.299068 (+ 60us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:11.299115 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.299193 (+ 78us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.568313 (+269120us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54002085d0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:11.568482 (+ 169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710509824}
W20260616 19:59:11.622099 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 60047.3us
I20260616 19:59:11.668336 8785 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:11.622787 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.630964 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.670434 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.638106 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.638209 8011 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:11.627192 10923 consensus_meta.cc:287] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.647s user 0.006s sys 0.017s
I20260616 19:59:11.642071 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.639631 10935 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:11.641098 8784 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:11.616051 10920 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:11.668679 8082 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:11.675951 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:11.694666 10882 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:11.670135 8043 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:11.695171 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:11.635720 10724 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.634138 10898 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:11.688320 8044 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:11.637153 10878 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:11.688788 10889 consensus_meta.cc:287] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.521s user 0.007s sys 0.012s
I20260616 19:59:11.693746 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.655488 10811 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.675691 10892 raft_consensus.cc:1081] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
W20260616 19:59:11.691955 10861 consensus_meta.cc:287] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.734s user 0.032s sys 0.039s
I20260616 19:59:11.698560 7934 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:11.617062 10888 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:11.716545 10952 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:11.702057 10946 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:11.703685 8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52458 (request call id 322) took 1215 ms. Trace:
W20260616 19:59:11.720065 10871 consensus_meta.cc:287] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.627s user 0.008s sys 0.002s
I20260616 19:59:11.700510 10951 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:11.642113 10868 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:11.744313 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:11.696242 10947 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:11.792352 10947 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:11.758666 10888 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.792795 10947 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:11.745919 10889 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.753580 10886 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260616 19:59:11.743731 10697 consensus_meta.cc:287] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 1.115s user 0.016s sys 0.016s
I20260616 19:59:11.762248 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.762691 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.805846 8041 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:11.762933 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.806313 8043 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
W20260616 19:59:11.750703 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 52341.8us
I20260616 19:59:11.807792 7934 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:11.749071 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 116778us
I20260616 19:59:11.810817 10647 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:11.711675 10558 consensus_meta.cc:287] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.603s user 0.019s sys 0.004s
I20260616 19:59:11.692780 10923 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.755824 8816 rpcz_store.cc:276] 0616 19:59:10.488078 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.489798 (+ 1720us) service_pool.cc:224] Handling call
0616 19:59:10.490100 (+ 302us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:10.511628 (+ 21528us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:10.515181 (+ 3553us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:10.520280 (+ 5099us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:10.520338 (+ 58us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.703449 (+1183111us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b540087f0d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:11.703503 (+ 54us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.703630 (+ 127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22949632}
I20260616 19:59:11.828588 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.829455 8114 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:59:11.704978 10695 consensus_meta.cc:287] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.661s user 0.018s sys 0.004s
W20260616 19:59:11.811685 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 170668us
W20260616 19:59:11.811968 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 116876us
I20260616 19:59:11.812774 8111 raft_consensus.cc:1240] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:11.806793 8036 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:11.810981 8085 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:11.818444 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.833287 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:11.821736 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.821769 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.834969 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:11.835570 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:11.823009 10724 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:11.823547 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:11.749212 10806 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:11.754029 10952 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:11.751266 10861 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.836678 8011 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:11.837509 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:11.845355 7964 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:11.837666 7968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:59:11.846184 7968 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:11.846567 7968 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:11.845821 8011 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:11.831578 10702 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:11.838166 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.839933 8119 raft_consensus.cc:2393] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:11.789656 10871 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.838866 10882 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:11.850696 10881 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:11.851043 10894 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.839033 10839 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:11.843182 10952 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:11.841693 10806 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:11.841354 10865 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:11.848651 10900 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:11.858314 7970 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:11.827997 10558 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.897646 10658 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.898121 10558 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.902302 8812 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:11.907549 8041 raft_consensus.cc:1240] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:11.907248 10891 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.840143 10902 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:11.909206 10902 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:11.909758 10902 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:11.911474 7966 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:11.837990 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:11.830677 10925 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:11.816082 10697 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.869184 10882 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:11.858201 10861 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.888742 7964 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:11.836141 10695 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:11.892786 10950 log.cc:927] Time spent T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260616 19:59:11.892951 10919 raft_consensus.cc:993] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
W20260616 19:59:11.893653 10955 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260616 19:59:11.849665 10702 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:11.861665 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.957260 10702 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:11.958330 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.895732 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.959862 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.961336 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.962677 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.913015 7933 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:11.913421 7963 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:11.907054 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 261) took 1765 ms. Trace:
I20260616 19:59:11.923072 10658 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.925652 10611 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:11.936271 8110 raft_consensus.cc:2393] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:11.880733 7968 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:11.970753 7968 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:11.976866 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 63964.5us
I20260616 19:59:11.977715 7963 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:11.978478 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.979305 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.917454 10893 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:11.943907 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.886176 10806 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:11.967159 7969 rpcz_store.cc:276] 0616 19:59:10.141843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.418532 (+1276689us) service_pool.cc:224] Handling call
0616 19:59:11.904274 (+485742us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.904524 (+ 250us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:11.904563 (+ 39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:11.904624 (+ 61us) raft_consensus.cc:1645] Marking committed up to 10
0616 19:59:11.904671 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:11.904747 (+ 76us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:11.906791 (+ 2044us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54003b1350. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:11.906965 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116180224}
W20260616 19:59:11.941843 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 104715us
I20260616 19:59:12.475358 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:12.475788 7934 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:11.981845 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:12.476280 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 435589us
I20260616 19:59:11.912734 10865 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:11.989157 10857 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:11.941520 10923 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:12.237043 10889 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.454499 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:12.022527 10942 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.497s user 0.003s sys 0.000s
I20260616 19:59:12.028790 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:12.030157 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.507496 8037 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:59:11.989877 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:12.384392 10887 consensus_meta.cc:287] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.735s user 0.009s sys 0.000s
I20260616 19:59:12.031560 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.457391 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.459199 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.040112 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.511062 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:12.204516 10868 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:12.131089 10697 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:59:12.462267 10871 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.464704 10888 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:12.038549 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:12.467617 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
W20260616 19:59:12.360993 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:12.036507 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2370) took 2375 ms. Trace:
I20260616 19:59:12.034713 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1719) took 3240 ms. Trace:
I20260616 19:59:12.520442 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.520524 7966 rpcz_store.cc:276] 0616 19:59:09.661535 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.291848 (+1630313us) service_pool.cc:224] Handling call
0616 19:59:11.292727 (+ 879us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.912066 (+619339us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:12.036236 (+124170us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b54003f8754. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:12.036437 (+ 201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422354176}
I20260616 19:59:12.039394 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2351) took 3321 ms. Trace:
I20260616 19:59:12.521889 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.479432 8011 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:12.522550 8041 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:12.520658 7967 rpcz_store.cc:276] 0616 19:59:08.794022 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.646094 (+1852072us) service_pool.cc:224] Handling call
0616 19:59:10.646379 (+ 285us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.632066 (+985687us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:12.034501 (+402435us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b540025a154. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:12.034659 (+ 158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2265490688}
I20260616 19:59:12.141747 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.484935 8820 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEARNER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260616 19:59:12.513710 10857 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:11.945576 10882 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:12.469213 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:12.093879 10879 consensus_meta.cc:287] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 1.031s user 0.016s sys 0.004s
I20260616 19:59:12.499235 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.446449 10891 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.501813 7961 raft_consensus.cc:2393] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:11.992712 10861 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:12.505010 7934 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:12.526954 7968 raft_consensus.cc:2393] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:12.527304 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.550478 7967 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 1.
I20260616 19:59:12.484449 10695 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:12.511103 8045 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 1.
I20260616 19:59:12.552862 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.552945 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.553422 8045 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:12.511893 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.554782 7961 raft_consensus.cc:1240] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3. Ops: []
I20260616 19:59:12.145052 8813 raft_consensus.cc:1240] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:12.460414 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.514205 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.515266 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.571862 8037 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:12.573004 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.503571 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:12.521186 8040 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:12.574982 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.097914 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.476442 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:12.523115 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 43767.4us
I20260616 19:59:12.480491 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.521842 7970 rpcz_store.cc:276] 0616 19:59:08.718117 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:10.561732 (+1843615us) service_pool.cc:224] Handling call
0616 19:59:10.562041 (+ 309us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:11.891477 (+1329436us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:12.039128 (+147651us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003fe654. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:12.039324 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2980540800}
W20260616 19:59:12.396463 10947 consensus_meta.cc:287] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.591s user 0.010s sys 0.000s
I20260616 19:59:11.989235 10702 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.519178 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:12.550565 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45637.9us
I20260616 19:59:12.526971 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:11.981981 10611 raft_consensus.cc:1081] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:12.596055 7964 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:12.527693 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.569792 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.576905 8011 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:12.594247 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.108392 10902 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.602511 10902 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.602972 10973 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:12.603286 8042 raft_consensus.cc:2393] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:12.543322 10935 consensus_meta.cc:287] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.841s user 0.007s sys 0.002s
I20260616 19:59:12.607028 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.605978 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.618939 8044 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:12.607823 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:12.569386 10897 consensus_meta.cc:287] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.899s user 0.009s sys 0.001s
I20260616 19:59:12.532198 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.516387 10887 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.549042 10871 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:12.526371 10857 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:12.626852 7968 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:12.569378 10696 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:12.604975 8816 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:12.630328 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:12.649688 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 118356us
I20260616 19:59:12.612453 8011 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:12.525911 10658 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260616 19:59:12.637530 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 171885us
W20260616 19:59:12.651803 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 47508.8us
I20260616 19:59:12.652767 8043 raft_consensus.cc:1240] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:59:12.660835 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.629276 10857 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:12.641237 8045 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:12.552098 10879 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:12.659159 10811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s user 0.013s sys 0.022s
I20260616 19:59:12.677946 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.679962 8041 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:12.593822 10947 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.626848 10897 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.684571 8044 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:12.657353 10932 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:12.686021 8811 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 6: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
W20260616 19:59:12.686373 10960 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.137s user 0.000s sys 0.000s
I20260616 19:59:12.686717 10865 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.689880 10806 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.625974 10935 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.706043 8036 raft_consensus.cc:1240] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:12.706553 8041 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:12.706648 10935 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.707006 10897 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:12.693817 10878 consensus_meta.cc:287] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.942s user 0.012s sys 0.000s
I20260616 19:59:12.708426 10878 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.711122 10878 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.697299 10887 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.697630 10702 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.646840 10891 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260616 19:59:12.698108 10969 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:12.699590 10947 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.700052 8038 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:12.639271 10871 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.702227 10879 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.679677 10868 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:12.664175 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.704756 8812 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9. Ops: []
W20260616 19:59:12.713732 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 118096us
I20260616 19:59:12.664973 10697 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:12.681078 10861 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:12.722611 8817 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:12.716972 10611 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:12.725792 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.726832 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.727532 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.726850 10611 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.726215 10966 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:12.729602 10966 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.728785 10697 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.731232 10871 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.731664 10966 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.725353 10868 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.733049 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.720079 10923 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:12.737332 10923 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.741796 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.742319 7964 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260616 19:59:12.722960 10952 consensus_meta.cc:287] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.845s user 0.008s sys 0.003s
I20260616 19:59:12.719975 8816 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:12.748716 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:12.754067 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.754359 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.756567 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.749171 10770 raft_consensus.cc:993] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:12.757602 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.758198 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.758754 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.746286 10952 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.762759 8812 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:12.764292 10952 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.764974 8115 raft_consensus.cc:1240] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:12.769884 8042 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:12.771013 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.778700 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.785174 10975 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260616 19:59:12.732550 8813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52474 (request call id 207) took 1041 ms. Trace:
I20260616 19:59:12.786369 10975 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:12.786845 10975 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:12.786424 8813 rpcz_store.cc:276] 0616 19:59:11.690835 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.691085 (+ 250us) service_pool.cc:224] Handling call
0616 19:59:11.691410 (+ 325us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:12.595558 (+904148us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:12.732345 (+136787us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54007d5854. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:12.732499 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809240320}
I20260616 19:59:12.790241 8814 raft_consensus.cc:1240] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:12.722048 10969 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.791242 10770 raft_consensus.cc:1081] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:12.749804 10885 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:12.792232 10885 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.792497 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.792984 8038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46718 (request call id 98) took 1035 ms. Trace:
I20260616 19:59:12.793309 8038 rpcz_store.cc:276] 0616 19:59:11.757714 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.757896 (+ 182us) service_pool.cc:224] Handling call
0616 19:59:11.928791 (+170895us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b5800011498. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:11.928982 (+ 191us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:12.721187 (+792205us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:12.792769 (+ 71582us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54003aa054. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:12.792931 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781141120}
I20260616 19:59:12.794073 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.794246 10885 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.794605 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.795434 8038 raft_consensus.cc:2393] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 6: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:12.795827 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.796921 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.795429 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:12.797830 7967 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:12.797930 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.798678 8041 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:12.799424 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.779690 8117 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 5.
I20260616 19:59:12.804636 8785 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:12.805837 8785 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:12.803311 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.806602 8037 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:12.794670 8819 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:12.802932 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.806627 10878 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:12.804733 8118 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:12.803660 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.801872 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.812688 10921 consensus_queue.cc:1059] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:12.813848 10921 raft_consensus.cc:3055] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:12.814028 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.814281 10921 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:12.814677 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.815290 8115 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:12.815137 10921 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.807261 10944 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:12.816519 10944 raft_consensus.cc:697] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:12.816639 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.816872 10697 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.842103 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.817085 8811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52458 (request call id 326) took 1256 ms. Trace:
I20260616 19:59:12.842638 10861 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.843109 8041 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:12.817440 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:12.817538 10944 consensus_queue.cc:237] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.799597 10871 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260616 19:59:12.834470 8117 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260616 19:59:12.813963 8044 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:12.845345 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.845669 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.846926 8044 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:12.787783 10975 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.848068 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.820832 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.815227 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.801502 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.850567 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.851529 10975 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:12.851589 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.825605 10868 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.827148 10857 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.853579 7962 raft_consensus.cc:1240] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:12.843104 8811 rpcz_store.cc:276] 0616 19:59:11.560987 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.566598 (+ 5611us) service_pool.cc:224] Handling call
0616 19:59:12.816873 (+1250275us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400204254. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:12.817006 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2438705536}
I20260616 19:59:12.850139 8112 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:12.854997 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.855646 8812 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:12.856458 10857 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.808990 8118 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:12.865054 8118 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.866181 8118 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:12.867424 8784 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:12.868216 10945 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:12.868594 10945 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:12.868640 8082 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:12.869014 10945 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:12.831161 10806 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:12.831692 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:12.831985 10923 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.832199 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.832885 10969 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.833933 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.834528 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.834831 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.878525 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.835937 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.837145 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.837049 10952 consensus_queue.cc:1059] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:59:12.884794 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.885623 10952 raft_consensus.cc:3055] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:12.885902 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.838395 10921 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:12.886607 8039 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 2.
I20260616 19:59:12.887418 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.883817 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.887929 10945 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:12.840077 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.889451 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.840605 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.879300 8040 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:12.837426 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.818689 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.844326 8115 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:12.838117 10611 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:12.820538 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:12.856050 8112 raft_consensus.cc:1240] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:12.814729 8110 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
W20260616 19:59:12.881042 10958 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.056s user 0.004s sys 0.027s
I20260616 19:59:12.883133 10868 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:12.883667 10878 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260616 19:59:12.885977 10952 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:12.891983 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.901739 8115 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:12.847154 8817 raft_consensus.cc:1240] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:12.896850 8118 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:12.808872 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:12.945937 7967 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:59:12.986368 8114 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:12.986784 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.987608 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.988385 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:12.958433 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.994445 8784 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:12.995429 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.996898 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.859083 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:12.997690 8784 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:12.998942 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.960335 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.000142 8038 raft_consensus.cc:2393] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:12.884637 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.003690 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.003988 10809 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:13.004354 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.005074 10809 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:13.005904 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.007498 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.859493 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.008265 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.009244 8044 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:13.009827 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 179658us
I20260616 19:59:12.897027 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.010473 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.968976 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.956590 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:12.969584 8043 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260616 19:59:12.990811 8118 raft_consensus.cc:1240] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:12.892453 10882 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:12.897794 10868 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:12.991860 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:13.001889 8814 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:12.954277 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:13.012121 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.025835 8044 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:13.011358 7934 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:13.002532 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:13.026903 8042 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:13.020184 8111 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:13.028548 10885 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:13.029096 10885 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:13.029503 10885 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:12.994531 8113 raft_consensus.cc:1240] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3. Ops: []
I20260616 19:59:12.966975 8041 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:13.032295 8117 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:13.032765 8040 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:13.001950 8039 raft_consensus.cc:2393] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:13.021003 10882 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:13.002477 10611 consensus_queue.cc:1059] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260616 19:59:13.019064 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:13.038647 10611 raft_consensus.cc:3055] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:13.019583 10868 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:13.039072 10611 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.040949 10611 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.042869 10611 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:13.013777 8045 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:13.027859 8037 raft_consensus.cc:2393] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 6: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:13.010207 8038 raft_consensus.cc:2393] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 3: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:13.067237 7965 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:13.070390 10949 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:12.905740 10952 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:12.896200 10958 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.071s user 0.005s sys 0.027s
I20260616 19:59:13.034672 7934 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:13.081849 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 47369.4us
W20260616 19:59:13.082582 10984 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.161s user 0.001s sys 0.000s
I20260616 19:59:13.082293 10949 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.084511 10976 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:13.085093 10976 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:59:13.085328 10985 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.164s user 0.000s sys 0.001s
I20260616 19:59:13.085558 10976 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.083695 7934 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:13.000666 8084 leader_election.cc:336] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.858s (SENT)
I20260616 19:59:13.000712 8784 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:13.072928 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:13.052098 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:13.014330 7968 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:13.101267 8042 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:13.070847 10945 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:13.109423 7962 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 6: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
W20260616 19:59:13.110081 10979 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.207s user 0.001s sys 0.000s
I20260616 19:59:13.111443 10959 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:13.112233 10923 consensus_queue.cc:1059] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:59:13.112300 10959 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:13.113569 10923 consensus_queue.cc:1243] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260616 19:59:13.114423 10969 raft_consensus.cc:3055] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:13.114943 10969 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.114650 10959 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.116415 10947 consensus_queue.cc:1059] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260616 19:59:13.118468 10947 raft_consensus.cc:3055] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:13.118966 10947 raft_consensus.cc:740] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:59:13.121786 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 121716us
I20260616 19:59:13.125090 10965 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:13.125643 10965 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:13.128301 10965 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Advancing to term 10
W20260616 19:59:13.129336 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:13.130059 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:13.131757 10880 consensus_queue.cc:1059] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:59:13.133112 10880 raft_consensus.cc:3055] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:13.133412 10880 raft_consensus.cc:740] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:13.134093 10868 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.134464 10880 consensus_queue.cc:260] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.135645 8116 raft_consensus.cc:2368] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7.
I20260616 19:59:13.135946 10880 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.091306 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:13.150467 10949 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:13.152490 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 152089us
I20260616 19:59:13.153855 10952 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:13.155551 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.121914 10947 consensus_queue.cc:260] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.162704 8815 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260616 19:59:13.163604 8784 leader_election.cc:400] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7.
I20260616 19:59:13.164022 8784 leader_election.cc:403] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:59:13.166242 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:13.162695 10947 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.129619 10969 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.173602 10969 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:13.162425 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 99) took 5121 ms. Trace:
I20260616 19:59:13.176210 10944 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:13.178232 10806 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:13.178689 8786 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:13.179963 10945 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:13.178890 10806 raft_consensus.cc:697] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:13.180570 10945 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:13.181329 10806 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.182708 7967 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:59:13.183537 7967 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:13.183995 7967 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.181890 10945 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.191982 7965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
I20260616 19:59:13.193053 7965 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:13.193637 7965 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:13.208715 10868 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:13.209347 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:13.175226 7866 rpcz_store.cc:276] 0616 19:59:08.040075 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.040256 (+ 181us) service_pool.cc:224] Handling call
0616 19:59:13.161185 (+5120929us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:11.446341 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:11.446925 (+ 584us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:11.446960 (+ 35us) write_op.cc:435] Acquired schema lock
0616 19:59:11.446980 (+ 20us) tablet.cc:662] Decoding operations
0616 19:59:11.447546 (+ 566us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:11.447619 (+ 73us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:11.447643 (+ 24us) tablet.cc:685] Acquiring locks for 3 operations
0616 19:59:11.447975 (+ 332us) tablet.cc:701] Row locks acquired
0616 19:59:11.447994 (+ 19us) write_op.cc:260] PREPARE: finished
0616 19:59:11.448130 (+ 136us) write_op.cc:270] Start()
0616 19:59:11.448267 (+ 137us) write_op.cc:276] Timestamp: P: 1781639951448080 usec, L: 0
0616 19:59:11.448288 (+ 21us) op_driver.cc:348] REPLICATION: starting
0616 19:59:11.448933 (+ 645us) log.cc:844] Serialized 1742 byte log entry
0616 19:59:11.824619 (+375686us) op_driver.cc:464] REPLICATION: finished
0616 19:59:12.673719 (+849100us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86c21682c 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe872d1dd70
0616 19:59:12.881466 (+207747us) write_op.cc:301] APPLY: starting
0616 19:59:12.881547 (+ 81us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:12.881678 (+ 131us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:12.881697 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:12.882173 (+ 476us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:12.882341 (+ 168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0616 19:59:12.882383 (+ 42us) write_op.cc:312] APPLY: finished
0616 19:59:12.882590 (+ 207us) log.cc:844] Serialized 35 byte log entry
0616 19:59:12.918975 (+ 36385us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:13.077112 (+158137us) write_op.cc:454] Released schema lock
0616 19:59:13.080813 (+ 3701us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:13.081703 (+ 890us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:13.081754 (+ 51us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":1056591,"apply.run_cpu_time_us":12251,"apply.run_wall_time_us":200098,"num_ops":3,"prepare.queue_time_us":367559,"prepare.run_cpu_time_us":39333,"prepare.run_wall_time_us":240870,"raft.queue_time_us":75914,"raft.run_cpu_time_us":32697,"raft.run_wall_time_us":1256707,"replication_time_us":376209,"spinlock_wait_cycles":15917952,"thread_start_us":1989866,"threads_started":5,"wal-append.queue_time_us":334854,"wal-append.run_cpu_time_us":14948,"wal-append.run_wall_time_us":1097885}]]}
I20260616 19:59:13.241515 7966 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260616 19:59:13.253547 7967 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.168823 7867 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: config changed from index 3 to 5, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129), VOTER de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130), VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131), VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:59:13.269026 10947 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:13.269769 10947 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.272833 10947 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:13.278842 7965 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:13.288426 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 125030us
I20260616 19:59:13.292488 7967 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:13.295095 7965 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:13.297685 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:13.236506 10965 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.308183 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:13.312500 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.313301 8115 raft_consensus.cc:3055] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:13.313680 8115 raft_consensus.cc:740] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.315182 10919 consensus_queue.cc:1059] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:13.314957 8115 consensus_queue.cc:260] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.316241 8115 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.317611 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.322988 10948 raft_consensus.cc:3055] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:13.323648 10948 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.324842 10948 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.326633 10948 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:13.331024 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:13.300758 10976 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.339253 7965 raft_consensus.cc:1240] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:13.339596 8116 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:13.339984 8116 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.339020 7867 catalog_manager.cc:5696] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 6 to 7, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 7 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:13.350715 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.359079 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:13.346692 10976 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:13.375631 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:13.365283 7867 catalog_manager.cc:5696] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 6 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:13.375638 8085 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:13.374823 8116 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:13.383785 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 79780us
I20260616 19:59:13.385058 8085 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:13.387743 10923 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
W20260616 19:59:13.388281 8084 leader_election.cc:336] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.599s (SENT)
I20260616 19:59:13.389096 10919 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:13.390113 10882 consensus_queue.cc:1059] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:13.393213 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.395499 7867 catalog_manager.cc:5696] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:13.406934 8116 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:13.410832 8084 leader_election.cc:336] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.662s (SENT)
I20260616 19:59:13.411530 8084 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:13.397100 10885 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.422469 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:13.424705 7968 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260616 19:59:13.413506 7867 catalog_manager.cc:5696] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 8 to 10, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 10 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:13.406677 8813 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
W20260616 19:59:13.428995 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 192853us
I20260616 19:59:13.429291 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:13.433995 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 46273.7us
I20260616 19:59:13.434319 10991 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:13.448624 10976 consensus_queue.cc:1059] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:59:13.450026 10976 raft_consensus.cc:3055] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:13.450435 10976 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:13.451077 10965 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:13.451820 10965 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260616 19:59:13.451349 10976 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.453675 10976 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:13.452593 7867 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 7 to 8, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 8 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } } }
I20260616 19:59:13.460888 8814 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:13.462333 7867 catalog_manager.cc:5696] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:59:13.479837 10885 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:13.482292 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
W20260616 19:59:13.483402 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:13.484227 7933 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:13.477406 8814 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260616 19:59:13.489282 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:13.490696 10885 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:13.490759 8011 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.491900 7867 catalog_manager.cc:5696] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 5 to 7, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:13.494823 8042 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260616 19:59:13.495668 10988 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.099s user 0.000s sys 0.002s
I20260616 19:59:13.495257 8085 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:13.499342 10949 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:13.504009 8037 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:13.503232 10944 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7.
W20260616 19:59:13.535441 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:13.543078 10887 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:13.517627 10991 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:13.543825 10887 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.548841 10923 raft_consensus.cc:697] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.520052 10944 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260616 19:59:13.556556 8116 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
W20260616 19:59:13.573202 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 78090.5us
W20260616 19:59:13.574362 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 108567us
W20260616 19:59:13.579658 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:13.545347 10949 consensus_queue.cc:1059] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260616 19:59:13.516726 10947 consensus_queue.cc:1059] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260616 19:59:13.585777 8115 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:13.585361 7934 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:13.590116 7934 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:13.591733 10989 log.cc:927] Time spent T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.157s user 0.002s sys 0.000s
I20260616 19:59:13.592038 10959 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:13.592640 10959 raft_consensus.cc:697] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:13.583034 10919 raft_consensus.cc:697] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.595153 10887 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:13.603276 8011 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.593262 10885 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:13.617245 8011 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.617537 10885 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:13.618546 10885 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.625679 7968 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260616 19:59:13.625980 8084 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:13.626232 7935 connection.cc:256] RPC call timeout handler was delayed by -1.1733351609999545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260616 19:59:13.626678 7935 connection.cc:256] RPC call timeout handler was delayed by -1.0968351609999445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260616 19:59:13.627517 10611 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:13.628785 10868 raft_consensus.cc:3055] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:13.628988 10945 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:13.629271 10868 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.629532 10945 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
W20260616 19:59:13.629801 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:13.627012 7935 connection.cc:256] RPC call timeout handler was delayed by -1.0842351610000378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260616 19:59:13.630173 7935 connection.cc:256] RPC call timeout handler was delayed by -1.0698351610000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260616 19:59:13.630514 7935 connection.cc:256] RPC call timeout handler was delayed by -1.0365351609999607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260616 19:59:13.630795 7935 connection.cc:256] RPC call timeout handler was delayed by -1.0320351609999534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260616 19:59:13.629442 10923 consensus_queue.cc:237] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:13.635915 10990 log.cc:927] Time spent T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.202s user 0.000s sys 0.002s
I20260616 19:59:13.635638 10945 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.635614 10921 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260616 19:59:13.602821 10993 consensus_queue.cc:1059] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260616 19:59:13.612449 10959 consensus_queue.cc:237] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.650514 8085 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:13.655622 8037 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
I20260616 19:59:13.641564 10865 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260616 19:59:13.656952 8084 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.657333 10991 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:13.658106 10991 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260616 19:59:13.658479 10991 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:13.658538 8084 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:13.658962 10991 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:13.659554 10992 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:13.663553 10947 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:13.659067 10919 consensus_queue.cc:237] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.664404 8117 raft_consensus.cc:1275] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:59:13.668890 10887 raft_consensus.cc:3055] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:13.669289 10887 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:13.669327 8815 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:13.670385 10887 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.671768 10887 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.671974 10947 consensus_queue.cc:1059] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260616 19:59:13.674520 10945 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:13.674535 10880 raft_consensus.cc:3055] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:13.675908 10880 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:13.674206 10947 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:13.676971 10947 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:13.677069 10880 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.677861 7968 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260616 19:59:13.677469 10868 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.679232 10947 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.680266 10806 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260616 19:59:13.642882 10949 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:59:13.681955 10993 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260616 19:59:13.681257 10947 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:13.675403 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:13.691013 8011 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.696979 10993 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:13.698522 8011 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:13.700542 11002 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:13.701313 11002 raft_consensus.cc:2804] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:59:13.701627 11002 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:13.701957 11002 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 1 FOLLOWER]: Advancing to term 2
W20260616 19:59:13.704598 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49652.6us
I20260616 19:59:13.705387 8085 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:13.700026 8011 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.720422 10993 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:13.733075 8011 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.734865 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.735918 10949 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:13.736784 10949 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:13.717013 10880 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:13.753036 8084 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:13.745365 10868 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.778548 8042 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260616 19:59:13.757098 10976 consensus_queue.cc:1059] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260616 19:59:13.780120 10980 raft_consensus.cc:3055] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:13.780560 10980 raft_consensus.cc:740] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:13.781621 10980 consensus_queue.cc:260] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.783233 10980 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:13.784633 8082 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:13.785044 10865 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260616 19:59:13.786126 10993 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:13.786722 10993 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:13.787710 10993 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:13.756587 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 51277.5us
I20260616 19:59:13.769706 8011 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.770267 10999 consensus_queue.cc:1059] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:59:13.771040 10923 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:13.772616 11001 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:59:13.774528 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 52832.3us
I20260616 19:59:13.802357 8008 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:13.756429 10991 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.812418 10991 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:13.815887 11002 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.839592 8085 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:13.842571 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 72958.1us
I20260616 19:59:13.847141 8084 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:13.856393 10997 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:13.858989 7935 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:13.860901 10921 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:13.861327 10921 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:13.861701 10921 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:13.853698 10923 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:13.874069 11002 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:13.846779 10949 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:13.876040 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:13.879474 8011 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.885179 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:13.884708 10868 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:13.890368 11004 raft_consensus.cc:3055] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:59:13.890872 11004 raft_consensus.cc:740] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:13.892305 10949 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:13.893869 7968 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:13.895278 7935 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:59:13.896930 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:13.899281 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 62456.2us
I20260616 19:59:13.900215 8084 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:13.903362 8784 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:13.903811 10923 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:13.904434 10945 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:13.907334 10949 consensus_queue.cc:1059] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:13.908828 10999 consensus_queue.cc:1059] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:13.909875 10999 raft_consensus.cc:3055] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:13.910172 10999 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:13.911142 10999 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:13.912330 11001 raft_consensus.cc:3055] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
W20260616 19:59:13.912422 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 58848.6us
I20260616 19:59:13.912812 11001 raft_consensus.cc:740] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:13.913648 11001 consensus_queue.cc:260] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.915011 11001 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.915550 8011 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:59:13.916391 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 161950us
I20260616 19:59:13.917181 8008 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:13.925156 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:13.925673 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:13.891817 11004 consensus_queue.cc:260] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:13.936511 11004 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:13.940685 8113 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:13.941329 10999 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:59:13.940227 10949 consensus_queue.cc:1175] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6827184e3cef42c2ada15b2d31afee4b" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260616 19:59:13.950330 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.959373 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:13.960085 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:13.964712 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:13.965701 8118 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:59:13.965718 8784 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.966946 8114 raft_consensus.cc:1240] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:13.962695 11002 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:13.968538 11002 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:13.968981 11002 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:13.969288 7967 raft_consensus.cc:1275] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260616 19:59:13.973459 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 47928.2us
I20260616 19:59:13.974498 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.953459 10980 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:13.946476 10945 consensus_queue.cc:1048] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260616 19:59:13.958369 10959 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:13.978735 8011 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:13.952870 10997 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260616 19:59:13.984274 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:13.998087 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:13.999562 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:14.000250 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.000921 7935 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.006912 8011 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:14.007725 10887 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:59:14.008584 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 91508.5us
I20260616 19:59:14.009513 8008 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:14.009994 11002 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.020610 8113 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260616 19:59:13.987211 10944 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260616 19:59:13.982086 10980 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:13.987879 10959 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:14.029263 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 63632.7us
W20260616 19:59:14.033098 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:14.050279 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44150.4us
I20260616 19:59:14.026446 10806 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260616 19:59:14.040593 10959 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:14.030946 10980 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.039913 10880 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:14.052464 10976 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260616 19:59:14.068930 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 62893.6us
I20260616 19:59:14.052284 10999 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:14.076764 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 69951.1us
I20260616 19:59:14.039732 10949 consensus_queue.cc:1059] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260616 19:59:14.081664 8008 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:14.072803 10976 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.101279 10976 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:14.103348 10921 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:14.098167 10949 consensus_queue.cc:1243] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260616 19:59:14.084590 8115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:14.077157 10806 consensus_queue.cc:1048] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260616 19:59:14.112416 8084 leader_election.cc:336] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 2.013s (SENT)
I20260616 19:59:14.125432 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.127426 8115 raft_consensus.cc:606] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:14.130353 7965 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260616 19:59:14.131522 10882 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:14.132108 11002 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.134672 11019 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260616 19:59:14.139598 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.112246 11001 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260616 19:59:14.140282 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.140401 8111 raft_consensus.cc:2393] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:14.141546 10997 consensus_queue.cc:1059] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
W20260616 19:59:14.144812 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 170614us
W20260616 19:59:14.146004 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 76952.8us
I20260616 19:59:14.161531 8008 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:14.165009 10921 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.168193 10997 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:14.168648 10887 raft_consensus.cc:3055] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:14.169093 10887 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:14.169140 10997 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.170010 10887 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.171450 10887 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:14.176585 8111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:14.177176 8111 raft_consensus.cc:606] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:14.177467 8111 raft_consensus.cc:614] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:14.178512 11002 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.179131 11002 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:14.181771 11002 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.183960 11004 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.184465 10997 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.185537 7935 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.186424 8111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:14.185945 11001 raft_consensus.cc:2764] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260616 19:59:14.188804 10947 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260616 19:59:14.190063 10880 raft_consensus.cc:3055] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:14.190492 10880 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:14.190356 11004 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.191359 10880 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.192910 11004 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.193650 11021 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:14.194430 7935 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.195297 11021 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:14.196811 7935 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.197631 10921 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:14.198185 10921 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.198187 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.198860 10921 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.199385 10975 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:14.200738 10976 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.202564 7935 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.203614 10959 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.204774 7962 raft_consensus.cc:1240] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:14.205818 10959 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.206696 10880 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.216328 8111 raft_consensus.cc:606] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:14.219676 8042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd"
dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
mode: GRACEFUL
new_leader_uuid: "6827184e3cef42c2ada15b2d31afee4b"
from {username='slave'} at 127.0.0.1:39112
I20260616 19:59:14.220328 8042 raft_consensus.cc:606] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.220729 8042 raft_consensus.cc:614] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:14.221601 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:14.223008 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260616 19:59:14.221658 8814 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260616 19:59:14.224422 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:14.225318 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:59:14.225842 10882 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260616 19:59:14.233767 10975 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:14.237834 7935 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.240396 10959 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:14.241029 10959 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.241598 10959 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:14.225841 11021 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:14.245255 11021 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:14.245765 11021 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.245940 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:14.190507 8008 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:14.248575 8008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 62299.1us
I20260616 19:59:14.248764 10997 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:14.249580 10997 raft_consensus.cc:697] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:14.250860 10997 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.253545 10980 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.255594 10980 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.202908 10976 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.258652 10921 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.246348 11021 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.239732 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.269426 10959 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:14.313458 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 180534us
I20260616 19:59:14.336251 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.336530 8045 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
W20260616 19:59:14.341703 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.345467 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.345579 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.346318 8040 raft_consensus.cc:2393] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:14.345505 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.347532 8044 raft_consensus.cc:2393] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:14.349987 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.349185 10980 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:14.351632 10980 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.352044 10980 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.352852 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.352878 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.353914 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.357837 8038 raft_consensus.cc:1240] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
W20260616 19:59:14.357801 8084 leader_election.cc:336] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 2.068s (SENT)
I20260616 19:59:14.358494 8111 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260616 19:59:14.360349 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:14.361538 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.369406 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.316430 10882 consensus_queue.cc:1059] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260616 19:59:14.344691 10887 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.349509 10947 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:14.374683 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.376740 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.377514 8041 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:14.378190 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 138626us
I20260616 19:59:14.378994 7935 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.385808 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.385833 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.336074 10995 consensus_queue.cc:1059] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260616 19:59:14.388801 10998 raft_consensus.cc:3055] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:14.389107 10959 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.389204 10998 raft_consensus.cc:740] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.389634 10947 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.390350 10998 consensus_queue.cc:260] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.392635 10998 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:14.397297 8039 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:14.401293 10887 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.402609 8045 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
W20260616 19:59:14.404935 11027 log.cc:927] Time spent T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.105s user 0.000s sys 0.001s
I20260616 19:59:14.407577 8038 raft_consensus.cc:1240] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 2. Ops: []
I20260616 19:59:14.407864 8815 raft_consensus.cc:1217] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 5.8->[6.9-6.9] Dedup: 6.9->[]
I20260616 19:59:14.408473 8815 raft_consensus.cc:1240] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260616 19:59:14.408829 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.409467 8044 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:14.410467 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.410854 10921 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.407634 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.408497 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.417330 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:14.418597 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 113623us
I20260616 19:59:14.418960 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.419224 10992 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:14.419831 10992 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.420437 10956 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:14.420656 8084 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.421490 10956 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:14.424705 7968 raft_consensus.cc:1275] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:59:14.425076 11029 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:14.427395 10975 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:14.427027 11029 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.427791 10975 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.428234 10975 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:14.429540 11031 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:14.425335 8041 raft_consensus.cc:2410] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260616 19:59:14.431797 7962 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:14.432967 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.433109 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.433758 8037 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:14.434144 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.434708 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:14.433766 8117 raft_consensus.cc:1275] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:59:14.444314 10992 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:14.445956 11020 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:59:14.448367 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:14.448559 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 69679.9us
I20260616 19:59:14.449767 7935 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.450943 11024 consensus_queue.cc:1059] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:59:14.451617 8117 raft_consensus.cc:1240] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 2. Current term is 3. Ops: []
I20260616 19:59:14.451985 11018 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:14.452245 7935 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.429549 11029 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.456385 10947 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.457839 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:14.458164 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:14.458714 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:14.462638 7961 raft_consensus.cc:1240] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 10. Current term is 11. Ops: []
I20260616 19:59:14.466228 8037 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
W20260616 19:59:14.468930 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 48378.2us
I20260616 19:59:14.476095 8084 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.470289 11018 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.484350 10921 consensus_queue.cc:1059] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:59:14.485854 11015 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:14.486263 10923 consensus_queue.cc:1048] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:59:14.486841 8085 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:14.495054 10887 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:14.496114 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 44009.6us
I20260616 19:59:14.497460 7961 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:14.497751 7962 raft_consensus.cc:1240] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:14.487390 10919 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:59:14.500272 10923 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:14.488325 10956 consensus_queue.cc:1059] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:14.469720 10880 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:14.490798 11029 raft_consensus.cc:3055] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:14.499207 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.484412 11018 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:14.505249 8084 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:14.508508 10948 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:14.513489 10993 raft_consensus.cc:3055] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:14.513429 10948 consensus_queue.cc:1059] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:14.513947 10993 raft_consensus.cc:740] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.514496 10948 raft_consensus.cc:3055] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:14.503454 10880 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.514890 10948 raft_consensus.cc:740] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.502157 10969 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260616 19:59:14.510996 10959 raft_consensus.cc:2764] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Leader pre-election decision vote started in defunct term 9: won
I20260616 19:59:14.517956 10991 raft_consensus.cc:3055] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Stepping down as leader of term 10
I20260616 19:59:14.518340 10991 raft_consensus.cc:740] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.519176 10991 consensus_queue.cc:260] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.519929 10993 consensus_queue.cc:260] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.515735 10948 consensus_queue.cc:260] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.521564 8042 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:59:14.524200 10880 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.524700 11021 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.526089 7963 raft_consensus.cc:1240] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:14.528367 11021 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:14.532828 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.534932 7963 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:14.536033 7855 catalog_manager.cc:5226] ChangeConfig:REMOVE_PEER RPC for tablet b14a29bdfa9541db9d3f1e93a225a342 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b022cac8cb5e46b7a39a748e81e257a7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:14.536851 11024 raft_consensus.cc:3055] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:59:14.537189 11024 raft_consensus.cc:740] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:14.537109 10956 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:14.537711 10956 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:14.538102 10956 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.538239 10965 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:14.538792 10965 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.542691 7964 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:59:14.546010 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.504981 11029 raft_consensus.cc:740] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:14.549664 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.549556 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.549685 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:14.550275 11029 consensus_queue.cc:260] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.550872 8115 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:14.551174 10965 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:14.551901 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:14.552451 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:14.552520 7964 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 5.
I20260616 19:59:14.553088 8041 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:14.538017 11024 consensus_queue.cc:260] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.562459 7935 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.562698 11029 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:14.563308 10995 raft_consensus.cc:993] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: : Instructing follower b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:14.563380 11024 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:14.563769 7935 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.564994 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.565474 10923 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:14.565855 11020 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:14.566269 11020 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.566629 11020 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.568269 10948 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.565869 10923 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.570714 10998 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:14.570493 10923 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.571177 10998 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.571650 10923 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:14.569551 10959 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:14.572301 10993 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:14.558481 11036 log.cc:927] Time spent T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
W20260616 19:59:14.576395 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 73896.2us
I20260616 19:59:14.577255 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.578190 10998 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:14.581332 10868 consensus_queue.cc:1059] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:14.623793 10956 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.624994 10868 raft_consensus.cc:3055] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:14.625453 10868 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.597101 8814 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260616 19:59:14.578680 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:14.629310 8011 leader_election.cc:336] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.909s (SENT)
I20260616 19:59:14.581806 10959 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:14.562763 8785 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:14.629853 10959 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.629638 11024 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260616 19:59:14.630100 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 76065.3us
I20260616 19:59:14.630378 10959 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.582309 10919 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:14.629681 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:14.580000 10882 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:14.583156 8009 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:14.563879 10995 raft_consensus.cc:1081] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Signalling peer b022cac8cb5e46b7a39a748e81e257a7 to start an election
I20260616 19:59:14.626329 10868 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.505574 10992 consensus_queue.cc:1059] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260616 19:59:14.637888 10868 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.638360 10992 raft_consensus.cc:3055] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:14.638809 10992 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.512678 10975 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.581136 7935 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:14.641309 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 60230.1us
I20260616 19:59:14.642187 10975 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.642867 10975 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:14.643162 7935 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.585539 11020 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.557281 11021 raft_consensus.cc:2804] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260616 19:59:14.557343 10991 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:14.606521 7864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55744 (request call id 220) took 5015 ms. Trace:
I20260616 19:59:14.614461 10980 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.615379 8815 raft_consensus.cc:1240] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:14.616693 7866 catalog_manager.cc:5696] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:59:14.579151 11018 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.643244 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.643415 10965 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:14.644590 7968 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:14.578737 8084 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.645406 7964 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790"
dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:49196
W20260616 19:59:14.645896 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 88243.5us
W20260616 19:59:14.647722 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 64647.2us
I20260616 19:59:14.646608 7864 rpcz_store.cc:276] 0616 19:59:09.590689 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:09.595697 (+ 5008us) service_pool.cc:224] Handling call
0616 19:59:14.606298 (+5010601us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16938,"reported_tablets":10}
I20260616 19:59:14.645548 11021 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:14.654928 11021 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 1 FOLLOWER]: Advancing to term 2
I20260616 19:59:14.655576 11001 consensus_queue.cc:1059] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:14.648478 10887 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:14.656659 11001 raft_consensus.cc:3055] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:14.656817 10887 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:14.657037 11001 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:14.657238 10887 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.657945 11001 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.659214 11001 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.651870 10965 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:14.660555 10965 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.649049 10880 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:14.652123 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.650765 7866 catalog_manager.cc:5696] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 8 to 9, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 9 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:59:14.653193 10921 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:14.670521 7867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 199) took 3535 ms. Trace:
I20260616 19:59:14.675803 7965 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
W20260616 19:59:14.654245 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 75575.6us
I20260616 19:59:14.654196 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:14.662031 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.675501 7962 raft_consensus.cc:1240] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:14.653371 11020 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.670042 7866 catalog_manager.cc:5696] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 8 to 9, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 9 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:14.681113 7866 catalog_manager.cc:5696] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 7 to 8, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 8 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:59:14.689932 7935 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.691385 7935 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.692371 11029 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:14.692864 11029 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:14.693650 11029 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.695262 10885 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:14.691918 11024 consensus_queue.cc:1059] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260616 19:59:14.696825 10975 raft_consensus.cc:3055] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:14.697175 10975 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:14.698004 10975 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.699333 10975 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.639726 10992 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.702376 10992 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.676307 7867 rpcz_store.cc:276] 0616 19:59:11.126321 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:11.168423 (+ 42102us) service_pool.cc:224] Handling call
0616 19:59:12.033302 (+864879us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:14.661963 (+2628661us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:13.695349 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:13.712872 (+ 17523us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:13.712920 (+ 48us) write_op.cc:435] Acquired schema lock
0616 19:59:13.712940 (+ 20us) tablet.cc:662] Decoding operations
0616 19:59:13.715847 (+ 2907us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:13.715949 (+ 102us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:13.715978 (+ 29us) tablet.cc:685] Acquiring locks for 8 operations
0616 19:59:13.862836 (+146858us) tablet.cc:701] Row locks acquired
0616 19:59:13.862854 (+ 18us) write_op.cc:260] PREPARE: finished
0616 19:59:13.863036 (+ 182us) write_op.cc:270] Start()
0616 19:59:13.863207 (+ 171us) write_op.cc:276] Timestamp: P: 1781639953862972 usec, L: 0
0616 19:59:13.863227 (+ 20us) op_driver.cc:348] REPLICATION: starting
0616 19:59:13.893138 (+ 29911us) log.cc:844] Serialized 3374 byte log entry
0616 19:59:13.895243 (+ 2105us) spinlock_profiling.cc:243] Waited 7 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e1208 00007fe872d1d401 00007fe872d1cbd8 00007fe872d21871 00007fe872d21829 00007fe872d217b9 00007fe872d21781 00007fe872d209ad 0000000000389354 0000000000389199
0616 19:59:14.086194 (+190951us) op_driver.cc:464] REPLICATION: finished
0616 19:59:14.306213 (+220019us) write_op.cc:301] APPLY: starting
0616 19:59:14.306306 (+ 93us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:14.306546 (+ 240us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:14.306591 (+ 45us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:14.307791 (+ 1200us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:14.308050 (+ 259us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0616 19:59:14.308098 (+ 48us) write_op.cc:312] APPLY: finished
0616 19:59:14.397264 (+ 89166us) log.cc:844] Serialized 65 byte log entry
0616 19:59:14.416998 (+ 19734us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:14.426438 (+ 9440us) write_op.cc:454] Released schema lock
0616 19:59:14.426484 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:14.426909 (+ 425us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:14.426945 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7,"reported_tablets":17,"spinlock_wait_cycles":544479616,"child_traces":[["op",{"apply.queue_time_us":212313,"apply.run_cpu_time_us":3434,"apply.run_wall_time_us":120685,"num_ops":8,"prepare.queue_time_us":31009,"prepare.run_cpu_time_us":7709,"prepare.run_wall_time_us":206857,"replication_time_us":206035,"spinlock_wait_cycles":16100736,"thread_start_us":193971,"threads_started":2}]]}
I20260616 19:59:14.704481 7935 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.706367 10885 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:14.707665 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.705431 10980 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.705009 11015 consensus_queue.cc:1059] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:59:14.708328 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 227) took 6003 ms. Trace:
I20260616 19:59:14.708802 7865 rpcz_store.cc:276] 0616 19:59:08.674744 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:08.674915 (+ 171us) service_pool.cc:224] Handling call
0616 19:59:14.669614 (+5994699us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39401,"reported_tablets":9}
I20260616 19:59:14.709056 11015 raft_consensus.cc:3055] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260616 19:59:14.709597 11015 raft_consensus.cc:740] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:14.713253 10980 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:14.715945 7935 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.718497 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.730078 8084 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.670931 10948 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:14.736507 7935 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.726656 10980 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:14.740185 11018 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.741158 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.743218 10965 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.733260 8814 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:14.744979 8082 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:14.745759 11038 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:14.746167 11038 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:14.746462 11038 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:14.747027 11015 consensus_queue.cc:260] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.728900 11021 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.749835 11021 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:14.750357 7964 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260616 19:59:14.750752 7964 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:14.757330 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:14.760751 10956 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:14.761439 10956 consensus_queue.cc:1059] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:59:14.762390 10995 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:14.763569 10995 raft_consensus.cc:3055] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:14.763916 10995 raft_consensus.cc:740] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.764181 10998 raft_consensus.cc:3055] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:14.736289 10993 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.764598 10998 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.764743 10995 consensus_queue.cc:260] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.765452 10998 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.766048 10995 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:14.766639 10998 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:14.769155 10885 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:14.772549 11015 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Advancing to term 3
W20260616 19:59:14.776269 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.762423 10956 consensus_queue.cc:1059] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:14.796310 7935 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.800266 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:14.802907 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 82451.4us
I20260616 19:59:14.804855 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:14.810864 7968 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.801618 10959 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:14.864707 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 68463.4us
W20260616 19:59:14.884554 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 72768.4us
I20260616 19:59:14.894874 8084 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:14.799538 10980 consensus_queue.cc:1059] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260616 19:59:14.897154 8084 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:14.887650 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.891342 11038 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.895177 10965 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:14.900905 11038 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:14.884708 7935 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:14.897755 10998 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:14.901638 10882 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:14.801843 10887 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.902271 10919 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:14.904220 10887 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:14.886821 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:14.908088 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:14.908751 8039 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.909862 7968 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:59:14.911255 10980 raft_consensus.cc:3055] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:14.911643 10980 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:14.912393 10980 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.913690 10980 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:14.915423 8039 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:59:14.916550 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:14.917155 8814 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:14.917853 10948 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:14.921350 10969 raft_consensus.cc:3055] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:14.921753 10969 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.923869 7964 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.929700 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:14.930402 8118 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:14.931028 7963 raft_consensus.cc:1240] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:14.930248 8785 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:14.937590 10806 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:14.922685 10969 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.938161 10806 raft_consensus.cc:697] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:14.939180 10806 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:14.939697 10969 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:14.941227 11029 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:14.941761 11029 raft_consensus.cc:697] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:14.942615 11029 consensus_queue.cc:237] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.947358 10921 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:14.947862 10921 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:14.951447 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 54382.2us
I20260616 19:59:14.953099 8084 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:14.954222 8118 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
W20260616 19:59:14.958431 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 73814.1us
I20260616 19:59:14.908897 10885 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:14.961066 8009 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:14.969486 7963 raft_consensus.cc:1240] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:14.969332 7968 raft_consensus.cc:1240] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 1. Current term is 3. Ops: []
I20260616 19:59:14.949167 10993 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:14.985380 8814 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:14.986465 8815 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:59:14.970731 10991 consensus_queue.cc:1059] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260616 19:59:14.988521 8082 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:14.990164 10947 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:14.991367 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:14.991603 10919 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:14.992406 7968 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:14.992781 10956 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:14.993034 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:14.993283 10956 raft_consensus.cc:697] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:14.994388 10947 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:14.994293 10956 consensus_queue.cc:237] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.994887 10947 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.995807 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:14.976959 10880 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:14.996647 10880 raft_consensus.cc:697] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:14.997450 10880 consensus_queue.cc:237] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:14.997874 10959 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.000116 10947 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.000679 10921 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.001389 7964 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.002938 7963 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3.
I20260616 19:59:15.004457 10887 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.004513 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.004989 10887 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.005843 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.007023 10887 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.015216 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.016120 7962 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:15.018704 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.021005 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.001164 11019 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.025377 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.026079 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.025343 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:15.028304 8110 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:15.025383 8118 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:15.032553 8009 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:15.034897 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:15.037997 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:15.048552 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.049772 7962 raft_consensus.cc:2393] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:15.029723 8115 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:15.033917 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:15.017333 10868 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.061283 8117 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 10.
I20260616 19:59:15.062791 8009 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:15.069589 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.026758 11038 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election won for term 8
W20260616 19:59:15.091936 8011 leader_election.cc:336] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.502s (SENT)
I20260616 19:59:15.094244 8011 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.065575 11001 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:15.068168 7934 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:15.089138 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.102766 7968 raft_consensus.cc:2393] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:15.070439 10947 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260616 19:59:15.106134 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 79973.9us
I20260616 19:59:15.059221 7963 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:15.107239 8084 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:15.056401 10948 consensus_queue.cc:1059] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260616 19:59:15.120446 11018 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260616 19:59:15.120770 11041 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.144s user 0.004s sys 0.000s
I20260616 19:59:15.121109 11018 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:15.121601 11018 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:15.061497 10959 consensus_queue.cc:1059] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260616 19:59:15.058401 11019 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.126113 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.102052 11001 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:15.095672 11038 raft_consensus.cc:697] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:15.066807 10868 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.140919 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.112880 10947 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:15.147069 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.135402 10887 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260616 19:59:15.151153 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:15.151957 7963 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:15.139396 10991 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:15.141788 11001 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:15.155507 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.146425 10947 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
W20260616 19:59:15.162190 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 55057.2us
I20260616 19:59:15.164052 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.165026 7968 raft_consensus.cc:2393] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:15.173557 11019 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.178234 10868 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:15.150167 10919 consensus_queue.cc:1059] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260616 19:59:15.180253 10885 raft_consensus.cc:3055] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:15.180723 10885 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:15.181559 10885 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.182813 10885 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:15.184595 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.184763 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.185376 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.185714 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.185984 7967 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:15.185845 11018 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.190112 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.186411 7966 raft_consensus.cc:2376] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6. Re-sending same reply.
I20260616 19:59:15.186892 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.187145 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.187662 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.185792 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.185388 7961 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:15.194551 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.194804 7969 raft_consensus.cc:2376] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 3. Re-sending same reply.
I20260616 19:59:15.196357 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.200595 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:15.201186 7966 raft_consensus.cc:2376] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7. Re-sending same reply.
I20260616 19:59:15.204015 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.204564 8037 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:15.206439 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.207923 8042 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260616 19:59:15.211024 8042 raft_consensus.cc:1275] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:59:15.211932 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.214867 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.215699 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.216346 7966 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:15.217337 7963 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:59:15.217712 8110 raft_consensus.cc:3055] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Stepping down as leader of term 1
I20260616 19:59:15.209688 8084 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:15.220894 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.218187 8110 raft_consensus.cc:740] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:15.221872 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.222908 7962 raft_consensus.cc:2368] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:15.223668 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.225360 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.225433 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.226127 8041 raft_consensus.cc:2393] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 2: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:15.226730 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.231438 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.231561 8040 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260616 19:59:15.232419 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.233067 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.232537 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.234511 8040 raft_consensus.cc:1275] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260616 19:59:15.235653 8786 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:15.234972 8043 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:15.237875 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.238896 8115 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260616 19:59:15.236544 11026 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:15.239990 8110 consensus_queue.cc:260] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.240540 11026 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:15.241003 11026 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:15.241135 8110 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 1 FOLLOWER]: Advancing to term 3
I20260616 19:59:15.200284 11038 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.242745 10887 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.243265 10887 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.246070 10887 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:15.246201 10880 consensus_queue.cc:1059] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:15.247220 10880 raft_consensus.cc:3055] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:15.247571 10880 raft_consensus.cc:740] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:15.247752 11001 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.248461 10880 consensus_queue.cc:260] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.249816 11001 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.240322 10868 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:15.249832 10880 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:15.253690 11018 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.257457 10806 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.257987 10806 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:15.261673 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 65501.8us
I20260616 19:59:15.263190 8084 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.264380 10998 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:59:15.265028 10998 raft_consensus.cc:697] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:15.265924 10998 consensus_queue.cc:237] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.266916 8110 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
W20260616 19:59:15.268602 10993 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 3 is higher.
I20260616 19:59:15.226011 10923 raft_consensus.cc:3055] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:15.280465 10923 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:15.281469 10923 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.283257 10923 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:15.272001 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:15.287072 8115 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:15.276257 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:15.288797 8117 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:15.239142 10959 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Committing config change with OpId 4.7: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:15.295351 7964 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:15.301613 10993 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:15.302332 10993 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.305004 10993 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:15.305915 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.290349 7934 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:15.308538 8037 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
W20260616 19:59:15.324052 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 45787.6us
I20260616 19:59:15.326493 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.332437 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:15.368046 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 121779us
I20260616 19:59:15.368121 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.260056 10806 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.370057 8037 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 5.
I20260616 19:59:15.370846 8011 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.369352 8041 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:15.376468 11057 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.378242 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.378818 8038 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 2 FOLLOWER]: Advancing to term 3
I20260616 19:59:15.387540 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:15.388010 8117 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:15.308090 10975 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.392500 10975 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.394889 10975 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:15.395670 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.396715 8110 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:15.401326 8115 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:15.360458 11018 raft_consensus.cc:2804] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:59:15.360067 10880 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.405390 7934 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:15.380080 11056 log.cc:927] Time spent T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.074s user 0.001s sys 0.000s
I20260616 19:59:15.413769 10997 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:15.308044 10919 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
I20260616 19:59:15.419591 8038 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:59:15.420681 10880 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.423751 10880 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.431602 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.319867 11015 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260616 19:59:15.431173 10980 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:15.440631 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.441309 8110 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 5.
I20260616 19:59:15.443034 10980 raft_consensus.cc:697] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:15.379498 10947 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.445482 8042 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260616 19:59:15.453469 8042 pending_rounds.cc:85] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 8
I20260616 19:59:15.446583 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.454030 8045 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:15.454792 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.448019 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.449414 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.450116 8043 raft_consensus.cc:1240] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260616 19:59:15.453004 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.455888 8009 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:15.455461 8036 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 8
I20260616 19:59:15.457208 8041 raft_consensus.cc:1275] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:59:15.426072 11018 raft_consensus.cc:697] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:15.453672 8042 pending_rounds.cc:107] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260616 19:59:15.460940 8042 raft_consensus.cc:2889] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:15.459231 8110 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260616 19:59:15.451932 8785 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:15.425846 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 55974.1us
I20260616 19:59:15.462538 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.463411 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.407706 11057 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.477614 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.486690 8039 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 10. Current term is 11. Ops: []
I20260616 19:59:15.497207 8084 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:15.457995 11038 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:15.460865 10880 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:15.524708 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:15.525328 7935 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:15.525345 8043 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:15.532944 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.551724 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:15.545123 8044 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:15.568075 7935 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:15.588982 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 54852.1us
I20260616 19:59:15.563586 8038 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:15.598121 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:15.598421 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 146556us
I20260616 19:59:15.598811 8044 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:15.527949 10998 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:15.618870 8039 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260616 19:59:15.640033 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 145606us
I20260616 19:59:15.541600 10806 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:15.539546 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:15.524610 11029 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260616 19:59:15.503340 10919 consensus_queue.cc:1048] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.104s
I20260616 19:59:15.506295 11055 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260616 19:59:15.600289 11021 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:15.599742 10995 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.599536 10969 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:15.649588 10885 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:15.650485 10806 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:15.670655 8084 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.511067 10980 consensus_queue.cc:237] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.526964 11038 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.619050 10956 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:15.559206 11018 consensus_queue.cc:237] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:15.728508 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 58047.7us
I20260616 19:59:15.729601 11061 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.729809 8084 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.730134 11061 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.732347 10998 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:15.732785 10998 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:15.733170 10998 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:15.728505 10969 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:15.738013 10969 raft_consensus.cc:697] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:15.738881 10969 consensus_queue.cc:237] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:15.740401 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 369618us
I20260616 19:59:15.525849 10921 consensus_queue.cc:1048] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.123s
I20260616 19:59:15.537055 10882 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260616 19:59:15.527715 11026 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.669876 8813 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:15.728067 8036 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:15.741204 8011 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.744513 8043 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:15.715193 8817 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:15.710089 10880 raft_consensus.cc:697] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:15.747814 10956 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:15.741940 10947 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.701476 10995 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:15.771525 7855 catalog_manager.cc:5226] ChangeConfig:REMOVE_PEER RPC for tablet 69c80791d0c44cb0b616bf402ae91b46 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b022cac8cb5e46b7a39a748e81e257a7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:15.773576 10995 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:15.764474 8011 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.765136 11071 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:15.753528 11057 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.765704 11026 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.722200 10806 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:15.751693 11061 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.746608 10997 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:15.843694 8042 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:59:15.844894 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.845672 8036 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:59:15.847119 8785 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:15.800752 10885 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.853444 7933 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:15.828028 11055 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:15.843115 11041 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260616 19:59:15.840704 11057 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260616 19:59:15.868037 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 103635us
W20260616 19:59:15.868134 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 137247us
W20260616 19:59:15.868005 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 300022us
I20260616 19:59:15.873507 8011 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.875835 8011 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.861361 11026 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260616 19:59:15.870563 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 221406us
I20260616 19:59:15.874500 7935 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:15.869612 11055 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.886560 10868 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.887086 10868 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.888298 11055 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.888707 10919 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:15.888875 11055 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:15.889302 10919 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.889767 11055 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.890820 11054 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:15.891244 11054 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:15.891499 10919 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:15.869688 10885 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.894169 10885 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.891844 11055 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:15.891592 11054 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:15.901670 7964 raft_consensus.cc:1275] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:59:15.903276 8011 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.880684 10997 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:59:15.915041 8042 raft_consensus.cc:1275] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260616 19:59:15.895408 10923 consensus_meta.cc:287] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.584s user 0.007s sys 0.000s
I20260616 19:59:15.905562 8011 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.919751 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.883800 11026 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:15.884114 11059 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:15.932339 8043 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:15.889184 10868 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:15.935153 8011 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.936760 8040 raft_consensus.cc:1275] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:59:15.940518 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.941538 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:15.942026 8110 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 10
I20260616 19:59:15.913391 11057 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:15.947126 8011 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:15.959736 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:16.018610 8042 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:15.962348 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:15.964267 8039 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:59:16.019934 8041 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:16.020341 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 149219us
I20260616 19:59:16.023159 8110 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
I20260616 19:59:15.915843 11071 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:16.026851 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 83947.4us
I20260616 19:59:16.025228 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.035808 8110 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:15.873425 10880 consensus_queue.cc:237] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:15.929204 11055 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Leader election lost for term 2. Reason: could not achieve majority
I20260616 19:59:16.032325 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:15.943572 11001 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:15.933192 11026 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:15.905203 11066 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260616 19:59:16.036821 8084 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.048224 7965 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:16.051699 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:16.062167 7962 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:16.063373 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:15.933259 11075 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:59:16.053188 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 178754us
I20260616 19:59:16.064983 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.065013 7935 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.020510 10997 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:16.055200 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:16.080670 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 154335us
I20260616 19:59:16.082813 7965 raft_consensus.cc:1275] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:59:16.082590 7967 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:16.084578 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 46909.4us
I20260616 19:59:16.085915 8784 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:16.087142 7962 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
I20260616 19:59:16.065689 7969 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
W20260616 19:59:16.089529 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 142434us
I20260616 19:59:16.090461 7935 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.090729 8011 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:16.093222 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.097350 7935 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.098865 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.099645 7966 raft_consensus.cc:2368] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 9. Current term is 10.
W20260616 19:59:16.099620 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 62909.7us
I20260616 19:59:16.049525 10947 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.064648 10959 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49182 (request call id 79) took 1515 ms. Trace:
I20260616 19:59:16.151798 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.065603 11038 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:15.947125 11059 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.080698 11070 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.106086 10975 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:16.106341 11021 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260616 19:59:16.088830 11074 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:16.180739 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 90069.8us
I20260616 19:59:16.186626 8118 raft_consensus.cc:1275] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:59:16.101648 10995 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.195696 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.103672 10956 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:16.157573 11076 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.088677 10969 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260616 19:59:16.157977 10959 rpcz_store.cc:276] 0616 19:59:14.532649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:14.532766 (+ 117us) service_pool.cc:224] Handling call
0616 19:59:14.543449 (+ 10683us) log.cc:844] Serialized 501 byte log entry
0616 19:59:16.047638 (+1504189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13555,"raft.run_cpu_time_us":1417,"raft.run_wall_time_us":1415,"thread_start_us":37351,"threads_started":1,"wal-append.queue_time_us":16978,"wal-append.run_cpu_time_us":1663,"wal-append.run_wall_time_us":1028000}
I20260616 19:59:16.196699 11070 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:16.241137 10998 consensus_meta.cc:287] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.507s user 0.007s sys 0.002s
I20260616 19:59:16.058154 11071 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.150171 10887 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:16.176281 11001 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:16.216292 11067 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.106583 11082 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:16.167788 11072 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:16.220530 10980 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:59:16.174392 10947 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.234917 11026 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.266290 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:16.306216 8043 raft_consensus.cc:2393] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:16.253374 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:16.234144 11059 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.276862 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:16.156822 10923 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:16.279768 8085 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:16.283517 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 177926us
I20260616 19:59:16.190739 10919 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:16.292959 7966 raft_consensus.cc:1275] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:59:16.263427 8011 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:16.283183 10991 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:16.312700 8085 leader_election.cc:400] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 9. Current term is 10.
I20260616 19:59:16.297396 10965 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:16.318049 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.328487 10965 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:16.329344 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:16.329509 10965 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.330750 7935 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.331607 10980 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:16.332842 10923 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.340622 10923 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:16.341771 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:16.342343 10947 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.344337 11026 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.345109 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:16.345724 8043 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:16.307920 11067 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.349637 11071 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.314203 10919 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:16.353552 10919 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:59:16.353833 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 90539.7us
I20260616 19:59:16.317368 11038 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:16.320097 11089 consensus_queue.cc:1048] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:16.354645 11038 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:16.355208 11038 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:16.314924 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:16.284835 10995 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.354687 8011 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:16.283445 11074 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.357240 10887 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:16.353466 10991 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.359472 10991 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:16.354722 11068 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader election won for term 3
I20260616 19:59:16.360344 11068 raft_consensus.cc:697] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:16.361323 11068 consensus_queue.cc:237] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.347998 11067 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.308362 11001 raft_consensus.cc:697] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:16.364957 10995 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.364898 11001 consensus_queue.cc:237] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.345789 11070 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.294601 11076 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.020994 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:16.366508 8043 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 7.
I20260616 19:59:16.318926 8085 leader_election.cc:403] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260616 19:59:16.372648 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 60145.4us
I20260616 19:59:16.388494 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 100) took 2559 ms. Trace:
I20260616 19:59:16.388664 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:16.391964 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 61287.2us
I20260616 19:59:16.393229 7935 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.395188 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.396659 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.397414 8117 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:16.398707 8009 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.395938 8118 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:16.405351 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.406116 8113 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:16.406764 7935 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.405341 8785 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.380513 10969 consensus_queue.cc:1048] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260616 19:59:16.409219 11072 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:16.409430 11074 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.409888 10956 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.404781 11089 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260616 19:59:16.411487 8009 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.412431 8009 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.412815 10947 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:16.413267 10947 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.413556 10997 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:16.413681 10947 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:16.414052 10997 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.414484 10997 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:16.410400 10956 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.381011 10991 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 9. Current term is 10.
I20260616 19:59:16.416239 11071 consensus_queue.cc:1048] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:59:16.408797 11067 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:16.417721 7935 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:16.419523 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.425429 8113 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:16.426658 8009 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.350322 10975 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:16.297200 10998 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.431015 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.389168 7866 rpcz_store.cc:276] 0616 19:59:13.858526 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:13.860294 (+ 1768us) service_pool.cc:224] Handling call
0616 19:59:13.880497 (+ 20203us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:16.388407 (+2507910us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:14.927716 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:14.990701 (+ 62985us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:14.990736 (+ 35us) write_op.cc:435] Acquired schema lock
0616 19:59:14.990753 (+ 17us) tablet.cc:662] Decoding operations
0616 19:59:14.991369 (+ 616us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:14.991433 (+ 64us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:14.991459 (+ 26us) tablet.cc:685] Acquiring locks for 4 operations
0616 19:59:14.991844 (+ 385us) tablet.cc:701] Row locks acquired
0616 19:59:14.991863 (+ 19us) write_op.cc:260] PREPARE: finished
0616 19:59:14.992002 (+ 139us) write_op.cc:270] Start()
0616 19:59:14.992113 (+ 111us) write_op.cc:276] Timestamp: P: 1781639954991953 usec, L: 0
0616 19:59:14.992132 (+ 19us) op_driver.cc:348] REPLICATION: starting
0616 19:59:14.992764 (+ 632us) log.cc:844] Serialized 1749 byte log entry
0616 19:59:15.175731 (+182967us) op_driver.cc:464] REPLICATION: finished
0616 19:59:15.370979 (+195248us) write_op.cc:301] APPLY: starting
0616 19:59:15.371070 (+ 91us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:15.371218 (+ 148us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:15.371322 (+ 104us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:15.371861 (+ 539us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:15.372046 (+ 185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0616 19:59:15.372110 (+ 64us) write_op.cc:312] APPLY: finished
0616 19:59:15.413099 (+ 40989us) log.cc:844] Serialized 41 byte log entry
0616 19:59:15.434002 (+ 20903us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:15.437774 (+ 3772us) write_op.cc:454] Released schema lock
0616 19:59:15.448917 (+ 11143us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:15.498339 (+ 49422us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:15.501737 (+ 3398us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3770,"reported_tablets":7,"spinlock_wait_cycles":19515136,"child_traces":[["op",{"apply.queue_time_us":194137,"apply.run_cpu_time_us":2511,"apply.run_wall_time_us":111397,"num_ops":4,"prepare.queue_time_us":199123,"prepare.run_cpu_time_us":3210,"prepare.run_wall_time_us":65326,"replication_time_us":168875,"thread_start_us":248282,"threads_started":2}]]}
I20260616 19:59:16.417234 10882 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:16.449502 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 349096us
W20260616 19:59:16.430294 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 64662.2us
I20260616 19:59:16.441927 11089 raft_consensus.cc:2804] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:16.453745 8011 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:16.434568 10806 consensus_meta.cc:287] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.658s user 0.003s sys 0.007s
I20260616 19:59:16.443451 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:16.423270 11067 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.428341 10868 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:16.456243 8786 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s user 0.000s sys 0.016s
W20260616 19:59:16.504966 8786 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s user 0.000s sys 0.016s
W20260616 19:59:16.505255 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 575479us
I20260616 19:59:16.432653 11075 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:16.442905 10975 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:16.501230 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 47571.6us
I20260616 19:59:16.509562 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:16.452615 10956 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.391141 11054 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.455418 10921 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:16.516762 8011 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:16.518397 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.519893 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:16.502667 11070 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:16.522573 11064 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:16.522656 11070 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:16.523208 11090 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:59:16.523496 11070 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:16.520501 8118 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:16.508041 11067 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:16.520615 8110 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 7
W20260616 19:59:16.527571 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 49436us
I20260616 19:59:16.506243 11089 raft_consensus.cc:697] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:16.508185 11075 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.511036 10975 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:16.517169 10923 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:16.481674 10882 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.513756 8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52470 (request call id 142) took 1109 ms. Trace:
I20260616 19:59:16.538882 11074 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.504552 10806 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.603238 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:16.603425 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.603660 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:16.603920 7965 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:16.604064 7964 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:16.512210 10868 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:16.598361 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 67283.9us
I20260616 19:59:16.611805 8110 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 7.
I20260616 19:59:16.528775 10887 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:16.617720 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.618052 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.626250 7964 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:16.626523 7969 raft_consensus.cc:2393] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:16.627185 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.627360 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.628073 7961 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:16.619462 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.630697 7965 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:16.620841 8011 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:16.621742 7962 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 6
I20260616 19:59:16.621737 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.536484 11075 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:16.618568 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.639942 7970 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:16.680608 7963 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:59:16.682596 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 61831us
I20260616 19:59:16.684418 8011 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:16.593853 10956 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.539508 11064 raft_consensus.cc:697] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:16.695854 10956 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.640558 11079 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.696882 11079 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.696918 11064 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.651993 11108 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.698159 11108 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.699218 11079 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.625806 10919 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:16.640372 11091 log.cc:927] Time spent T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.091s user 0.001s sys 0.000s
I20260616 19:59:16.700325 10998 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.701546 10919 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.650167 11089 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:16.702121 10919 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:16.702802 11054 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:16.702623 10919 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.704524 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:16.705145 8819 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:16.718412 7962 raft_consensus.cc:1275] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:59:16.640008 11099 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.725159 8044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2439) took 1262 ms. Trace:
I20260616 19:59:16.668495 11076 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.613929 11066 consensus_queue.cc:1048] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260616 19:59:16.640682 11086 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.592916 11090 consensus_queue.cc:1059] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.133s
I20260616 19:59:16.600777 10923 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.693848 10995 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.704981 11071 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260616 19:59:16.729624 8044 rpcz_store.cc:276] 0616 19:59:15.449610 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:15.606600 (+156990us) service_pool.cc:224] Handling call
0616 19:59:15.618145 (+ 11545us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:15.834776 (+216631us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:15.834825 (+ 49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:15.850685 (+ 15860us) log.cc:844] Serialized 26 byte log entry
0616 19:59:16.050020 (+199335us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:16.080343 (+ 30323us) log.cc:844] Serialized 14 byte log entry
0616 19:59:16.103101 (+ 22758us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:16.103188 (+ 87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:16.359493 (+256305us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b18003e0260. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c233a33 00007fe86c235e01 00007fe86c235db9 00007fe86c235d49 00007fe86c235d11 00007fe86c23500d 0000000000389354 0000000000389199 00007fe86c216fba
0616 19:59:16.705366 (+345873us) raft_consensus.cc:1685] finished
0616 19:59:16.711830 (+ 6464us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:16.711986 (+ 156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40132,"prepare.queue_time_us":36909,"prepare.run_wall_time_us":4977,"raft.queue_time_us":167088,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":54286,"spinlock_wait_cycles":61597184,"thread_start_us":380301,"threads_started":2,"wal-append.queue_time_us":318002}
I20260616 19:59:16.732555 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.599236 11082 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
I20260616 19:59:16.613160 11074 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.737888 10919 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:16.741750 11103 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.092s user 0.001s sys 0.000s
I20260616 19:59:16.738518 11112 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:16.717734 11108 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.724897 10887 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:16.737522 11099 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.745121 10806 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.745584 8819 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 11.
I20260616 19:59:16.746613 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.788023 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:16.788360 8110 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:16.788661 8117 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:16.747596 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.750541 10868 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.750880 11086 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.741788 10882 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.724823 11079 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:16.786038 10919 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:16.800449 11079 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:16.800727 10919 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.599375 8815 rpcz_store.cc:276] 0616 19:59:15.395994 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:15.404576 (+ 8582us) service_pool.cc:224] Handling call
0616 19:59:15.433841 (+ 29265us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:15.458068 (+ 24227us) raft_consensus.cc:1534] Early marking committed up to index 8
0616 19:59:15.867134 (+409066us) log.cc:844] Serialized 14 byte log entry
0616 19:59:16.434341 (+567207us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b180037d260. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86ee8c85c 00007fe86c216c0b 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c230a95 00007fe86f2dd440 00007fe86f2efaf6 00007fe86f300667
0616 19:59:16.445520 (+ 11179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:16.449596 (+ 4076us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:16.452214 (+ 2618us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:16.459067 (+ 6853us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:16.505638 (+ 46571us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3012,"prepare.queue_time_us":548393,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":60444,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":83706,"spinlock_wait_cycles":8572288,"thread_start_us":519572,"threads_started":2}
I20260616 19:59:16.754221 10997 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.791944 10887 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.777026 10995 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.790571 8009 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.805410 11099 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:16.759181 7970 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:16.805949 10995 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.803050 11075 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.801044 11079 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:16.803534 10997 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.806682 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:16.785346 11084 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.419s user 0.003s sys 0.011s
I20260616 19:59:16.812567 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.813262 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:16.791020 11018 raft_consensus.cc:3055] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Stepping down as leader of term 10
I20260616 19:59:16.797849 11074 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.821888 8113 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:16.808115 11090 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.819406 11068 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:16.808982 11072 raft_consensus.cc:476] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Not starting pre-election -- already a leader
I20260616 19:59:16.807322 10921 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.803073 10919 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.833307 8009 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:16.856501 10921 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.836957 8117 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:59:16.822427 11108 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.864286 11108 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.865005 8784 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:16.866267 11064 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:16.867203 11064 raft_consensus.cc:697] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:16.867642 11075 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.868366 11075 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:16.868827 11075 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.868412 11064 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.869289 11075 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:16.873919 11086 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.875773 10887 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.885739 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:16.910218 11086 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:16.910883 11086 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.914002 11086 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.914242 11075 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.916452 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.917236 8117 raft_consensus.cc:2393] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 7: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:16.923027 7855 catalog_manager.cc:5209] ChangeConfig:REMOVE_PEER RPC for tablet f24cd55ee50d4bcfaee34b2adaa9b08a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260616 19:59:16.923918 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:16.924592 8119 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:16.925292 8113 raft_consensus.cc:1275] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:59:16.927522 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:16.916009 8118 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:16.825719 11018 raft_consensus.cc:740] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:16.811086 10947 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.937278 11015 raft_consensus.cc:476] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Not starting pre-election -- already a leader
I20260616 19:59:16.789422 11112 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.944628 10923 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.944708 11112 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.847481 11090 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.947687 11090 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.948625 10997 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:16.949167 10997 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.951885 10997 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:16.952864 11113 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:16.953428 11113 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.955049 11070 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:16.955591 11070 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.956825 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:16.957592 11026 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:16.957703 11070 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:16.958473 11038 consensus_meta.cc:287] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.603s user 0.009s sys 0.003s
I20260616 19:59:16.958881 11038 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.959079 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:16.958118 11026 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.839977 11068 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:16.938225 11072 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:16.961826 10995 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:16.961283 10868 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:16.942379 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.964412 11067 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.966464 11067 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.966567 10947 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:16.958979 10923 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:59:16.961963 11072 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:16.968902 8044 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:59:16.969209 11072 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:16.970750 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:16.963438 11018 consensus_queue.cc:260] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:16.972048 11018 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:16.978015 11038 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.961251 11068 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:16.963862 11026 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:16.983927 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:16.984920 8045 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
W20260616 19:59:16.988338 8037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51870 (request call id 2429) took 1543 ms (client timeout 1622 ms). Trace:
W20260616 19:59:16.988804 8037 rpcz_store.cc:269] 0616 19:59:15.445109 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:15.445242 (+ 133us) service_pool.cc:224] Handling call
0616 19:59:16.988092 (+1542850us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000b59d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:16.988246 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2394200448}
I20260616 19:59:16.995009 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:16.995814 8040 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:16.996975 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:16.998864 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:16.984025 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:17.000206 8112 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 11
I20260616 19:59:17.002889 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:17.003194 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.004931 8113 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260616 19:59:17.000121 8110 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:16.919759 11075 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:17.015599 8009 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:17.019066 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.116470 8113 raft_consensus.cc:1275] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:59:17.117738 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:17.118137 11105 log.cc:927] Time spent T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.446s user 0.002s sys 0.000s
I20260616 19:59:17.118348 8113 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
W20260616 19:59:17.121690 10975 consensus_meta.cc:287] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.525s user 0.009s sys 0.000s
I20260616 19:59:17.124544 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:17.125541 8110 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:17.122154 10975 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.064035 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.186918 8112 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 11.
I20260616 19:59:17.368988 8045 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:59:17.071316 11038 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.299s
I20260616 19:59:17.014366 11082 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:17.017858 11090 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:17.002161 11113 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:17.373175 11082 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.012537 10980 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:17.374691 10980 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.376315 11082 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:17.203070 8084 leader_election.cc:336] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.622s (SENT)
I20260616 19:59:17.377117 8084 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:17.233281 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:17.020007 7965 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260616 19:59:17.379832 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 176810us
I20260616 19:59:17.021988 8115 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:17.219422 11057 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:17.380205 8043 raft_consensus.cc:2955] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:17.035946 8044 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260616 19:59:17.381335 11057 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:17.382872 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:17.383319 11057 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:17.386380 8113 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
W20260616 19:59:17.055706 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:17.024379 7961 raft_consensus.cc:1240] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:17.233122 11092 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.150s
I20260616 19:59:17.377219 10980 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:17.204481 10919 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:17.391182 10919 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.391790 11080 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:17.392019 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:17.392345 11080 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:17.063942 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:17.388995 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.393967 7962 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:17.394508 11080 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:17.394937 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.395560 7962 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:17.373615 11090 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:17.389957 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 374442us
W20260616 19:59:17.132567 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:17.393354 10919 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:17.389233 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.410320 7934 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:17.396672 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.413955 7962 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:17.414594 8110 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:17.397729 11082 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:59:17.398489 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.417759 7961 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:17.418785 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.419813 7961 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:17.420651 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:17.421379 7961 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:17.399385 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.422219 7965 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:17.367555 10868 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:17.063313 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.429298 7969 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:17.430401 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.431021 7969 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:17.403218 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:17.414842 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.433985 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.431897 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:17.435093 7969 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:17.397106 7964 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:17.439566 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.412130 7966 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 11.
I20260616 19:59:17.440992 11083 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:17.441589 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.441538 11083 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.398506 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:17.397542 8038 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:17.401715 11001 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:17.134081 10959 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:17.455077 8817 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260616 19:59:17.455808 7934 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:17.422662 11108 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:17.427578 11055 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:17.457450 11055 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:17.457809 11055 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:17.327697 11068 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.460803 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:17.461663 8813 raft_consensus.cc:2410] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260616 19:59:17.461787 11068 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:17.427235 11086 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:17.462572 7969 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:17.462702 11086 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.122294 11079 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.423837 8085 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:17.411494 8009 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:17.442242 7970 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 7.
I20260616 19:59:17.442539 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:17.413179 10975 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:17.421377 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:17.401767 11038 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:17.497567 8038 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 11.
W20260616 19:59:17.447420 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 48988.5us
I20260616 19:59:17.400633 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.409916 10923 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:17.452211 7966 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:59:17.423807 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:17.464757 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.424829 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.506999 8041 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:17.507114 8036 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
W20260616 19:59:17.475420 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 64014us
I20260616 19:59:17.510130 11068 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:59:17.501802 8044 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:17.400115 11113 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:17.476217 11108 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:17.476641 11089 consensus_queue.cc:1048] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260616 19:59:17.477645 11083 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:17.466131 11122 log.cc:927] Time spent T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.069s user 0.000s sys 0.001s
I20260616 19:59:17.475191 10959 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.464371 10868 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260616 19:59:17.500315 11038 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.500775 11086 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:17.502422 7935 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:17.475970 11123 log.cc:927] Time spent T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260616 19:59:17.503844 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.503921 7961 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:17.505164 7934 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:17.544981 8820 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:17.577605 10995 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:17.475814 11001 raft_consensus.cc:697] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:17.494719 11075 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:17.580199 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.583350 10921 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:17.548333 10923 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:17.584681 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:17.578167 11125 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260616 19:59:17.547323 11015 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:17.579262 10919 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:17.577913 10969 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:17.568261 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 144524us
I20260616 19:59:18.064798 8817 raft_consensus.cc:3060] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:18.064601 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:18.071489 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:17.755290 8111 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:18.072618 8085 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.073592 8085 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.074971 8038 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:18.076637 8036 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
W20260616 19:59:18.063364 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 639633us
I20260616 19:59:18.070812 8043 raft_consensus.cc:2368] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7.
I20260616 19:59:17.754205 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:17.584549 11108 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.084175 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 1998) took 1067 ms. Trace:
I20260616 19:59:18.085817 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.086624 8042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 1995) took 1087 ms. Trace:
I20260616 19:59:18.084203 8085 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.087777 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:18.101624 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 599223us
I20260616 19:59:18.102306 8044 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:18.090212 11068 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:18.091605 11109 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 1.441s user 0.000s sys 0.023s
I20260616 19:59:18.093215 10923 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:18.095942 10956 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:18.085505 10995 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.097379 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.091616 11038 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:18.084836 8045 rpcz_store.cc:276] 0616 19:59:17.016385 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:17.753785 (+737400us) service_pool.cc:224] Handling call
0616 19:59:18.083958 (+330173us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:18.101262 11055 consensus_meta.cc:287] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.643s user 0.012s sys 0.000s
I20260616 19:59:18.088340 8036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 1994) took 1078 ms. Trace:
I20260616 19:59:18.153203 8036 rpcz_store.cc:276] 0616 19:59:16.998975 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:17.555668 (+556693us) service_pool.cc:224] Handling call
0616 19:59:18.077354 (+521686us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.087028 8042 rpcz_store.cc:276] 0616 19:59:16.999366 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.085544 (+1086178us) service_pool.cc:224] Handling call
0616 19:59:18.086567 (+ 1023us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:18.125478 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 620332us
I20260616 19:59:18.155656 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:18.087726 11108 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:18.157857 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:18.085222 10868 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:18.148401 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51870 (request call id 2452) took 1125 ms. Trace:
W20260616 19:59:18.137892 8786 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.626s (SENT) [suppressed 38 similar messages]
I20260616 19:59:18.069653 11089 raft_consensus.cc:476] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260616 19:59:18.155560 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.101651 11083 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:18.133200 11112 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader election won for term 11
I20260616 19:59:18.103222 8085 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.102725 7935 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:18.165261 8039 rpcz_store.cc:276] 0616 19:59:17.023331 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.097116 (+1073785us) service_pool.cc:224] Handling call
0616 19:59:18.148340 (+ 51224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.173167 8111 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260616 19:59:18.195379 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 92715.6us
I20260616 19:59:18.195468 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.196739 8039 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:18.197729 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.198865 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.199507 8039 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
W20260616 19:59:18.199538 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 96400.2us
I20260616 19:59:18.200644 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:18.200892 8085 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.201351 8039 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:18.178208 8038 raft_consensus.cc:2468] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:18.137892 11086 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:17.906162 10998 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.208284 10969 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:18.208724 10969 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.209074 10969 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:18.207921 10998 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.214658 10998 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:18.218729 11110 log.cc:927] Time spent T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 1.578s user 0.012s sys 0.000s
I20260616 19:59:18.160326 11055 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.221751 11055 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:18.179934 11083 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.191376 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.222563 11083 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:18.222772 8043 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:18.223718 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.224314 8043 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:18.225067 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.225611 8043 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:18.191700 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:18.227066 10959 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:18.226356 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:18.227988 8043 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:18.192996 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:18.203744 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2175) took 1273 ms. Trace:
I20260616 19:59:18.228410 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:18.109580 11114 consensus_queue.cc:1059] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260616 19:59:18.228739 8111 rpcz_store.cc:276] 0616 19:59:16.922813 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:16.924236 (+ 1423us) service_pool.cc:224] Handling call
0616 19:59:17.754872 (+830636us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b580019f598. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:17.755128 (+ 256us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.195790 (+440662us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.196004 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6208256}
I20260616 19:59:18.230003 11114 raft_consensus.cc:3055] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:18.202888 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:18.202119 8085 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.207118 11078 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:18.203540 11086 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.190369 8817 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260616 19:59:18.153975 10923 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:18.170881 11082 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.233307 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:18.235610 11015 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.255686 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2487) took 1155 ms. Trace:
I20260616 19:59:18.258651 11136 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.237488 8041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 1981) took 1283 ms. Trace:
I20260616 19:59:18.193606 11079 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:18.259419 11136 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.260468 11142 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:18.172606 10868 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.260989 11142 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.261641 10868 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:18.262522 11136 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:18.148507 11068 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.231976 8114 raft_consensus.cc:1275] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260616 19:59:18.268536 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2178) took 1344 ms. Trace:
I20260616 19:59:18.241835 8085 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.244124 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:18.109030 11129 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:18.245411 8817 pending_rounds.cc:85] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Aborting all ops after (but not including) 8
I20260616 19:59:18.273365 8817 pending_rounds.cc:107] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.9
I20260616 19:59:18.274112 8817 raft_consensus.cc:2917] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Aborting config change with OpId 8.9: Aborted: Op aborted by new leader
I20260616 19:59:18.242209 11021 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:18.274880 8817 raft_consensus.cc:2889] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:18.236864 10969 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.250576 8038 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:59:18.258787 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 120922us
I20260616 19:59:18.110234 11075 raft_consensus.cc:697] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:59:18.254179 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 190637us
I20260616 19:59:18.285917 8113 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:18.259364 8041 rpcz_store.cc:276] 0616 19:59:16.953678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:16.956581 (+ 2903us) service_pool.cc:224] Handling call
0616 19:59:18.237303 (+1280722us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54003abbd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:18.237432 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263539584}
I20260616 19:59:18.256227 7968 rpcz_store.cc:276] 0616 19:59:17.094648 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:17.094849 (+ 201us) service_pool.cc:224] Handling call
0616 19:59:18.233159 (+1138310us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:18.249631 (+ 16472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14806912}
I20260616 19:59:18.258136 11015 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:18.245756 11078 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:18.197633 7855 catalog_manager.cc:5226] ChangeConfig:REMOVE_PEER RPC for tablet 69c80791d0c44cb0b616bf402ae91b46 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b022cac8cb5e46b7a39a748e81e257a7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:18.268931 8114 rpcz_store.cc:276] 0616 19:59:16.923787 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:16.924252 (+ 465us) service_pool.cc:224] Handling call
0616 19:59:18.231469 (+1307217us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b580019f598. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:18.231659 (+ 190us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.268227 (+ 36568us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.268473 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4749696}
I20260616 19:59:18.246755 10921 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260616 19:59:18.270216 11090 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:18.246536 11086 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:18.243432 11112 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:18.239437 11114 raft_consensus.cc:740] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:18.242714 8039 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:18.286135 8009 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:18.315371 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 119153us
I20260616 19:59:18.308348 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:18.283687 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:18.293298 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.317617 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 1991) took 1322 ms. Trace:
I20260616 19:59:18.318387 8041 raft_consensus.cc:2393] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:18.318255 8039 rpcz_store.cc:276] 0616 19:59:16.995466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.316693 (+1321227us) service_pool.cc:224] Handling call
0616 19:59:18.316996 (+ 303us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.317210 (+ 214us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.317235 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:18.317277 (+ 42us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:18.317324 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.317403 (+ 79us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:18.317547 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.320086 8041 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260616 19:59:18.320721 8039 raft_consensus.cc:1275] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260616 19:59:18.320878 8041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 1993) took 1324 ms. Trace:
I20260616 19:59:18.321578 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 1996) took 1320 ms. Trace:
I20260616 19:59:18.321432 8041 rpcz_store.cc:276] 0616 19:59:16.996053 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.319408 (+1323355us) service_pool.cc:224] Handling call
0616 19:59:18.319798 (+ 390us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:18.320597 (+ 799us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.320815 (+ 218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.322290 8039 rpcz_store.cc:276] 0616 19:59:17.001265 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.319915 (+1318650us) service_pool.cc:224] Handling call
0616 19:59:18.320246 (+ 331us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.321297 (+ 1051us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.321520 (+ 223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.323208 8041 raft_consensus.cc:1240] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260616 19:59:18.245231 10998 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:18.298355 8085 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:18.324352 10998 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.324978 10998 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:18.325681 8117 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:18.247205 8043 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 6.
I20260616 19:59:18.280853 10956 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260616 19:59:18.327967 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46718 (request call id 138) took 1315 ms. Trace:
I20260616 19:59:18.328331 8043 rpcz_store.cc:276] 0616 19:59:17.012795 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.226164 (+1213369us) service_pool.cc:224] Handling call
0616 19:59:18.327909 (+101745us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.252728 10923 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.280944 11021 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:18.254571 11082 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.244794 11038 consensus_queue.cc:1059] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260616 19:59:18.329659 8043 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:59:18.346632 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2450) took 1329 ms. Trace:
I20260616 19:59:18.347054 8043 rpcz_store.cc:276] 0616 19:59:17.019350 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.329061 (+1309711us) service_pool.cc:224] Handling call
0616 19:59:18.329345 (+ 284us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.346385 (+ 17040us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.346577 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.323941 8041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 1999) took 1307 ms. Trace:
I20260616 19:59:18.349375 8041 rpcz_store.cc:276] 0616 19:59:17.016770 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.322586 (+1305816us) service_pool.cc:224] Handling call
0616 19:59:18.322944 (+ 358us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:18.323654 (+ 710us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.323883 (+ 229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.297084 11078 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:18.336884 8038 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:18.324293 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2000) took 1307 ms. Trace:
I20260616 19:59:18.252977 11001 consensus_queue.cc:237] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.351859 8039 rpcz_store.cc:276] 0616 19:59:17.017113 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.323278 (+1306165us) service_pool.cc:224] Handling call
0616 19:59:18.323615 (+ 337us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.323840 (+ 225us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.323880 (+ 40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:18.323940 (+ 60us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:18.323990 (+ 50us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.324075 (+ 85us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:18.324228 (+ 153us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:18.300921 7855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 109639us
I20260616 19:59:18.357256 11112 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:18.292627 11055 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260616 19:59:18.341926 11021 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:18.358387 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2484) took 1335 ms. Trace:
I20260616 19:59:18.360023 10923 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:18.331103 10956 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:18.360625 11142 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:18.364902 11137 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.365315 11133 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:18.373420 11139 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:18.389055 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 96845.6us
I20260616 19:59:18.389530 7967 rpcz_store.cc:276] 0616 19:59:17.022540 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:17.022718 (+ 178us) service_pool.cc:224] Handling call
0616 19:59:18.099456 (+1076738us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b5800004e98. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:18.099679 (+ 223us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:18.100014 (+ 335us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.100044 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:18.100489 (+ 445us) log.cc:844] Serialized 26 byte log entry
0616 19:59:18.296910 (+196421us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:18.296951 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.297029 (+ 78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:18.358080 (+ 61051us) raft_consensus.cc:1685] finished
0616 19:59:18.358104 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:18.358310 (+ 206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2972,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":27326,"spinlock_wait_cycles":4537216,"thread_start_us":196058,"threads_started":1,"wal-append.queue_time_us":201876}
I20260616 19:59:18.368121 11079 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:18.340370 11090 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.397428 8084 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:18.415269 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 129175us
I20260616 19:59:18.620515 8036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2472) took 1234 ms. Trace:
I20260616 19:59:18.842159 8009 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:18.842338 8036 rpcz_store.cc:276] 0616 19:59:17.386494 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.619427 (+1232933us) service_pool.cc:224] Handling call
0616 19:59:18.619836 (+ 409us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.620053 (+ 217us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.620090 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:18.620146 (+ 56us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:18.620213 (+ 67us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.620286 (+ 73us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:18.620430 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:18.454777 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:18.844476 8036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2474) took 1420 ms. Trace:
I20260616 19:59:18.844878 8036 rpcz_store.cc:276] 0616 19:59:17.424323 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.843758 (+1419435us) service_pool.cc:224] Handling call
0616 19:59:18.844025 (+ 267us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:18.844203 (+ 178us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.844223 (+ 20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:18.844258 (+ 35us) raft_consensus.cc:1645] Marking committed up to 9
0616 19:59:18.844285 (+ 27us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.844334 (+ 49us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:18.844426 (+ 92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:18.503959 11082 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:18.846058 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:19.023022 11158 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:19.069621 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:19.039235 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:18.981401 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:19.033432 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:19.088742 8115 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:19.061877 8038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2469) took 1676 ms. Trace:
W20260616 19:59:19.018147 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 176102us
I20260616 19:59:19.092757 8009 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:19.093835 8009 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:19.095456 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:18.950980 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:18.510118 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:19.017457 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:18.982956 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:19.097481 8110 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:19.010972 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:19.099340 7963 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:59:18.588577 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 291795us
W20260616 19:59:18.979599 8043 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s user 0.006s sys 0.003s
W20260616 19:59:19.100072 8043 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s user 0.007s sys 0.003s
I20260616 19:59:19.070176 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:18.945007 8111 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s user 0.004s sys 0.000s
W20260616 19:59:19.101374 8111 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s user 0.005s sys 0.002s
I20260616 19:59:19.102388 7970 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
W20260616 19:59:19.075066 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 677671us
I20260616 19:59:18.982610 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:19.104738 8112 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:19.078626 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:18.393502 10969 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:19.089588 8038 rpcz_store.cc:276] 0616 19:59:17.385736 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.357522 (+971786us) service_pool.cc:224] Handling call
0616 19:59:18.359088 (+ 1566us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:18.365456 (+ 6368us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.365506 (+ 50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:18.366064 (+ 558us) log.cc:844] Serialized 26 byte log entry
0616 19:59:18.672700 (+306636us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:18.672738 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:18.672908 (+ 170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:19.061549 (+388641us) raft_consensus.cc:1685] finished
0616 19:59:19.061580 (+ 31us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:19.061790 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18158,"raft.queue_time_us":598594,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":36154,"thread_start_us":306223,"threads_started":1,"wal-append.queue_time_us":692928}
I20260616 19:59:19.190969 8115 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:19.142257 8085 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:19.442775 8011 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:19.097637 8040 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:19.124768 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:19.285508 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:19.444129 7969 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:18.398322 10923 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
W20260616 19:59:19.362375 8010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.010s sys 0.009s
W20260616 19:59:18.786335 10868 consensus_meta.cc:287] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.524s user 0.008s sys 0.002s
I20260616 19:59:19.195241 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:18.450541 10995 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:18.398754 11134 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:19.403405 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:19.142715 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:19.444833 8786 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s user 0.007s sys 0.019s
W20260616 19:59:19.483974 8786 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s user 0.007s sys 0.019s
W20260616 19:59:19.484488 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 635386us
I20260616 19:59:19.485244 8786 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:19.441818 8084 leader_election.cc:400] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7.
I20260616 19:59:19.486142 8084 leader_election.cc:403] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:59:19.486910 7961 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:19.254585 8113 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:59:19.487615 8113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 254) took 1091 ms. Trace:
W20260616 19:59:18.772403 11108 consensus_meta.cc:287] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.588s user 0.011s sys 0.000s
W20260616 19:59:18.465023 11131 log.cc:927] Time spent T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.261s user 0.000s sys 0.001s
I20260616 19:59:18.394657 10956 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Advancing to term 12
W20260616 19:59:18.496117 11109 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.102s user 0.000s sys 0.000s
I20260616 19:59:19.023108 10921 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:18.421870 11136 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:19.050374 11150 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:19.195632 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:19.532801 7963 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:59:19.449751 8010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s user 0.010s sys 0.010s
I20260616 19:59:19.533263 7934 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:19.124182 11147 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:19.023782 11144 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:19.023620 11142 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:18.848618 11083 consensus_meta.cc:287] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.626s user 0.012s sys 0.002s
I20260616 19:59:18.848698 10959 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.465023 11138 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.515588 11068 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:19.218484 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:19.023527 11153 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:18.450515 11129 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:19.286610 11074 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:19.456506 8119 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:19.032653 11029 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:19.218899 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:18.790720 11021 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.451138 11137 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.300572 11157 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.224s
I20260616 19:59:19.124701 11079 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.753655 11078 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.394616 11075 consensus_queue.cc:237] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.462467 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:18.466981 11114 consensus_queue.cc:260] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.539331 10921 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.041000 11077 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.418s
I20260616 19:59:19.465093 11161 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:19.427325 11159 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:18.750691 11086 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.466459 11163 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.945533 11090 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.459213 10923 raft_consensus.cc:697] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:19.476193 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2470) took 2090 ms. Trace:
I20260616 19:59:19.455781 10969 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:19.481017 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2188) took 1207 ms. Trace:
I20260616 19:59:19.296110 11149 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:18.699687 10998 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.459744 10868 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:18.792771 11015 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.445380 11001 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:19.440575 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:19.443617 8040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51870 (request call id 2461) took 2060 ms (client timeout 1555 ms). Trace:
I20260616 19:59:19.487906 8113 rpcz_store.cc:276] 0616 19:59:18.395778 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.395922 (+ 144us) service_pool.cc:224] Handling call
0616 19:59:19.253819 (+857897us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b580019f598. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:19.254109 (+ 290us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:19.487365 (+233256us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:19.487561 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38999424}
W20260616 19:59:19.488543 8785 connection.cc:256] RPC call timeout handler was delayed by -1.0379855910000515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260616 19:59:19.495712 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 353558us
W20260616 19:59:19.526603 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 84986.5us
I20260616 19:59:18.821676 11038 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:19.456876 11151 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:19.541062 7934 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:19.542407 7963 raft_consensus.cc:1275] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260616 19:59:19.488819 10995 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:18.954026 11133 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.496656 11082 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:19.550526 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 107817us
I20260616 19:59:19.531935 11136 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:19.539397 10885 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:19.579097 7967 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:19.143770 11158 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.655287 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2503) took 2158 ms. Trace:
I20260616 19:59:19.543787 11147 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:19.527561 11108 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:19.579629 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 94432.8us
I20260616 19:59:19.688964 11029 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:18.945392 11139 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.712731 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 256) took 1316 ms. Trace:
I20260616 19:59:19.713280 11173 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:19.728605 8116 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260616 19:59:19.489151 11134 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.730234 8118 raft_consensus.cc:2393] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:19.737026 10969 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.723354 8039 rpcz_store.cc:276] 0616 19:59:17.385970 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.390723 (+1004753us) service_pool.cc:224] Handling call
0616 19:59:18.391184 (+ 461us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:18.826215 (+435031us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.826257 (+ 42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:18.955069 (+128812us) log.cc:844] Serialized 26 byte log entry
0616 19:59:19.294279 (+339210us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:19.294341 (+ 62us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:19.294439 (+ 98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:19.475964 (+181525us) raft_consensus.cc:1685] finished
0616 19:59:19.475984 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:19.476117 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3090,"raft.queue_time_us":629018,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":136,"thread_start_us":296620,"threads_started":1,"wal-append.queue_time_us":469615}
I20260616 19:59:19.550999 11142 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.706207 11074 raft_consensus.cc:697] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:59:19.736690 8037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51870 (request call id 2464) took 2352 ms (client timeout 1935 ms). Trace:
I20260616 19:59:19.557160 11083 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:19.736337 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 193297us
I20260616 19:59:19.741638 8084 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:19.543520 11150 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.575671 10959 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.743848 7962 raft_consensus.cc:2393] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:20.238071 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1827) took 1156 ms. Trace:
W20260616 19:59:19.739135 8040 rpcz_store.cc:269] 0616 19:59:17.383241 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.509824 (+1126583us) service_pool.cc:224] Handling call
0616 19:59:19.443552 (+933728us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:20.239404 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:19.758351 11077 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:19.759311 11176 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7.
I20260616 19:59:19.759680 11021 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:19.759763 11165 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:19.722980 11161 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.760493 11168 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:19.760885 11160 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:19.722761 11129 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.312532 11079 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:19.548815 11144 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.666297 11153 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.578315 11138 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.759429 11172 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:19.725622 8111 rpcz_store.cc:276] 0616 19:59:18.273848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.275820 (+ 1972us) service_pool.cc:224] Handling call
0616 19:59:18.284281 (+ 8461us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:18.333097 (+ 48816us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:18.333133 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:18.351444 (+ 18311us) log.cc:844] Serialized 26 byte log entry
0616 19:59:19.101844 (+750400us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:59:19.101903 (+ 59us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:19.101997 (+ 94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:19.480775 (+378778us) raft_consensus.cc:1685] finished
0616 19:59:19.480802 (+ 27us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:19.480948 (+ 146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80021,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":736189,"threads_started":2,"wal-append.queue_time_us":1059152}
I20260616 19:59:19.728490 11163 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:19.940898 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 210413us
I20260616 19:59:19.991595 7965 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:59:20.212018 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 471327us
W20260616 19:59:19.917158 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:19.737407 11149 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.214300 8116 pending_rounds.cc:85] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 8
W20260616 19:59:20.214563 8037 rpcz_store.cc:269] 0616 19:59:17.384424 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.735049 (+2350625us) service_pool.cc:211] Skipping call since client already timed out
0616 19:59:19.736606 (+ 1557us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260616 19:59:19.798070 7964 rpcz_store.cc:276] 0616 19:59:17.419123 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:17.453978 (+ 34855us) service_pool.cc:224] Handling call
0616 19:59:17.466592 (+ 12614us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:17.466971 (+ 379us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:17.467007 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:17.489887 (+ 22880us) log.cc:844] Serialized 26 byte log entry
0616 19:59:17.807640 (+317753us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:17.807701 (+ 61us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:17.807803 (+ 102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:19.577141 (+1769338us) raft_consensus.cc:1685] finished
0616 19:59:19.577165 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:19.577336 (+ 171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":20451,"thread_start_us":227957,"threads_started":1,"wal-append.queue_time_us":626025}
I20260616 19:59:20.215898 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:20.227144 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 686121us
I20260616 19:59:19.810575 8036 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:20.212989 8114 rpcz_store.cc:276] 0616 19:59:18.396601 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.396736 (+ 135us) service_pool.cc:224] Handling call
0616 19:59:19.711539 (+1314803us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b580019f598. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6a3e 000000000036f859 00007fe8742348e0 00007fe8742fd7c4 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:19.711723 (+ 184us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:19.711908 (+ 185us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:19.711950 (+ 42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:19.711996 (+ 46us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:19.712026 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:19.712094 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:19.712668 (+ 574us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19841280}
W20260616 19:59:20.231161 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 476750us
I20260616 19:59:20.150791 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1812) took 1020 ms. Trace:
I20260616 19:59:20.233464 11159 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:20.245362 11188 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:20.246320 11197 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:20.247216 8085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s user 0.012s sys 0.000s
I20260616 19:59:20.248791 11190 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:20.238399 7962 rpcz_store.cc:276] 0616 19:59:19.081332 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.195931 (+114599us) service_pool.cc:224] Handling call
0616 19:59:20.238014 (+1042083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:19.794550 11136 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:19.879444 11157 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:19.798851 10980 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:20.311103 11193 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.311749 11195 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.314107 8038 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:59:20.314795 7966 raft_consensus.cc:2393] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:20.219523 11147 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:19.753046 11038 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.251086 11077 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:19.803608 10885 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.793409 11082 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:19.989940 10923 consensus_queue.cc:237] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.312885 11160 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:19.762604 11151 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:20.710103 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:20.710641 7934 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:20.749325 8039 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:20.751960 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 2006) took 1288 ms. Trace:
W20260616 19:59:20.750252 8085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.009s user 0.020s sys 0.019s
I20260616 19:59:20.750674 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.751291 8116 pending_rounds.cc:107] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.9
I20260616 19:59:20.257198 11173 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:20.567194 8041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2455) took 3185 ms. Trace:
W20260616 19:59:20.752658 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 1.01101e+06us
I20260616 19:59:20.210618 11181 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:20.789070 8116 raft_consensus.cc:2917] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Aborting config change with OpId 8.9: Aborted: Op aborted by new leader
I20260616 19:59:20.790311 8116 raft_consensus.cc:2889] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260616 19:59:20.759371 8786 leader_election.cc:336] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.626s (SENT)
I20260616 19:59:20.157946 11171 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.752476 8039 rpcz_store.cc:276] 0616 19:59:19.463771 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:20.215600 (+751829us) service_pool.cc:224] Handling call
0616 19:59:20.751904 (+536304us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:20.365794 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 135428us
W20260616 19:59:20.705446 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 963838us
I20260616 19:59:20.748667 7968 rpcz_store.cc:276] 0616 19:59:19.014828 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.014959 (+ 131us) service_pool.cc:224] Handling call
0616 19:59:19.015316 (+ 357us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:19.069023 (+ 53707us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:19.069077 (+ 54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:19.069552 (+ 475us) log.cc:844] Serialized 26 byte log entry
0616 19:59:19.493700 (+424148us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:19.695432 (+201732us) log.cc:844] Serialized 14 byte log entry
0616 19:59:19.909547 (+214115us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:19.909660 (+ 113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:20.035174 (+125514us) raft_consensus.cc:1685] finished
0616 19:59:20.035208 (+ 34us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:20.035418 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7608,"prepare.queue_time_us":264026,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":244484,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":71995,"thread_start_us":838757,"threads_started":3,"wal-append.queue_time_us":480340}
I20260616 19:59:20.749838 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:20.787768 8009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.008s user 0.013s sys 0.010s
W20260616 19:59:20.326982 11169 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
W20260616 19:59:20.373266 10995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s user 0.004s sys 0.027s
I20260616 19:59:20.158932 11182 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:20.788861 11173 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:20.810593 8037 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:20.791318 8786 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:20.437538 11078 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:20.811365 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.811295 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.696379 11133 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:20.812520 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 62786us
I20260616 19:59:20.564198 11071 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.671689 11134 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.752645 11077 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:20.658294 11180 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:20.793551 8041 rpcz_store.cc:276] 0616 19:59:17.381843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.368501 (+986658us) service_pool.cc:224] Handling call
0616 19:59:18.368824 (+ 323us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:19.729217 (+1360393us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:19.729253 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:19.729296 (+ 43us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:19.729329 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:20.566802 (+837473us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400292f54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:20.566873 (+ 71us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:20.567074 (+ 201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3128151424}
I20260616 19:59:20.520706 11086 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:20.488119 10921 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:20.855712 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 43634.3us
I20260616 19:59:20.856535 8011 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:20.508834 11176 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:20.851162 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 2009) took 1386 ms. Trace:
I20260616 19:59:20.327589 11021 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.605384 11015 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.605064 10998 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:20.686761 11090 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.791849 11189 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.626587 10868 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:20.863234 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 152652us
I20260616 19:59:20.865097 11078 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:59:20.865108 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 152668us
I20260616 19:59:20.313210 11183 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.851528 8039 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:20.926741 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51870 (request call id 2477) took 1424 ms. Trace:
W20260616 19:59:20.799139 8009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s user 0.013s sys 0.010s
I20260616 19:59:20.927109 8039 rpcz_store.cc:276] 0616 19:59:19.503218 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:20.793270 (+1290052us) service_pool.cc:224] Handling call
0616 19:59:20.926684 (+133414us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:20.752429 11001 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.949s user 0.002s sys 0.017s
I20260616 19:59:20.810642 10980 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:20.757898 11157 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.810438 10995 raft_consensus.cc:697] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:20.873646 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:20.937484 8041 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:20.937594 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.851982 11079 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:20.874608 11137 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:20.909415 8045 rpcz_store.cc:276] 0616 19:59:19.464539 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:20.751705 (+1287166us) service_pool.cc:224] Handling call
0616 19:59:20.851100 (+ 99395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:20.857023 11181 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.947547 8036 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:20.947661 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:20.948459 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 189104us
I20260616 19:59:20.949048 11181 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.949234 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.859448 11029 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:20.859624 11133 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:20.950703 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.860450 11083 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:20.809854 11188 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:20.914757 11202 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:20.914958 11174 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.611s user 0.001s sys 0.000s
I20260616 19:59:20.810544 11190 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.440865 11074 consensus_queue.cc:237] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:20.811031 11175 log.cc:927] Time spent T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.784s user 0.001s sys 0.000s
W20260616 19:59:20.919126 7935 connection.cc:256] RPC call timeout handler was delayed by -1.066615564000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
I20260616 19:59:20.920468 11153 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.871191 11158 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.870780 11161 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.924386 8785 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:20.812532 11193 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.875463 11129 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.878113 11165 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:20.989231 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 64947.6us
I20260616 19:59:20.956130 11144 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:20.873057 11134 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:20.809872 11197 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.933357 7934 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:20.909138 8084 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:20.995257 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 86218.4us
I20260616 19:59:20.871227 11199 consensus_queue.cc:1059] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
I20260616 19:59:20.946038 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.327939 11168 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.000072 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:20.950852 11079 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:20.915844 11086 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:21.000702 8042 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:20.949976 8045 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:21.003135 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.922127 11018 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:20.953243 8044 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:21.004663 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:20.938728 11001 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.135s user 0.003s sys 0.034s
I20260616 19:59:20.811106 11172 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.993260 11158 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:20.814232 11195 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:20.866887 11182 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.858188 11171 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:20.969135 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:20.971130 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 59273.7us
I20260616 19:59:21.040521 8011 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:21.041971 8011 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:21.031785 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 45491us
I20260616 19:59:21.047076 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2451) took 4027 ms. Trace:
I20260616 19:59:21.047684 8040 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 11.
I20260616 19:59:21.047648 8043 rpcz_store.cc:276] 0616 19:59:17.022842 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.348021 (+1325179us) service_pool.cc:224] Handling call
0616 19:59:18.348459 (+ 438us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:19.100549 (+752090us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:19.100603 (+ 54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:19.439838 (+339235us) log.cc:844] Serialized 26 byte log entry
0616 19:59:19.546246 (+106408us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:19.546299 (+ 53us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:19.546488 (+ 189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:21.046783 (+1500295us) raft_consensus.cc:1685] finished
0616 19:59:21.046811 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:21.046993 (+ 182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38250,"raft.queue_time_us":485677,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":126719,"thread_start_us":832105,"threads_started":2,"wal-append.queue_time_us":290134}
I20260616 19:59:20.916363 11021 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.049976 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:20.978694 11149 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:20.932308 11072 consensus_queue.cc:1048] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260616 19:59:20.949873 10921 consensus_queue.cc:1048] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260616 19:59:20.938306 8039 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:20.964901 11082 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.939715 11078 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.995981 8084 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:21.000857 11134 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:20.996694 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 63407.7us
I20260616 19:59:20.866925 11071 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.005208 8045 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:20.986604 10980 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:21.005244 8044 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:21.093338 7934 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:20.984845 11133 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:21.104601 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:20.940330 11189 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:20.950699 11183 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.113567 11168 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.963653 11160 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:20.982808 11163 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:20.913654 11180 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:20.924965 11159 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.028625 11138 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:21.029839 11193 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.031015 11197 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.004447 11083 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:21.001825 11055 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:21.047250 8111 raft_consensus.cc:1275] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:59:21.051117 8115 raft_consensus.cc:2393] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:21.016237 10885 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:21.013298 11001 raft_consensus.cc:697] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:21.055218 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.018244 11151 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.065089 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.013139 11086 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.092614 11209 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:21.053617 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 63676.9us
I20260616 19:59:21.064481 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:21.143998 8043 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:21.053001 8011 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:20.971611 11211 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:21.070827 8042 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:21.000231 11201 consensus_queue.cc:1048] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.193s
I20260616 19:59:21.073576 8041 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:20.987159 11202 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:20.995607 11165 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.077498 11171 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.080428 11195 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.013521 11018 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.071921 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:21.159133 8113 raft_consensus.cc:2368] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
I20260616 19:59:21.052726 11137 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.017491 11204 consensus_queue.cc:1048] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.150s
I20260616 19:59:21.014613 11015 consensus_queue.cc:1059] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260616 19:59:20.908839 11176 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:59:21.089620 7933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.107s user 0.021s sys 0.025s
W20260616 19:59:21.161506 7933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.179s user 0.021s sys 0.025s
I20260616 19:59:21.090591 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:21.091284 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 95382us
W20260616 19:59:21.096426 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:21.097970 7934 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:21.051112 11038 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:21.034257 11158 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.053432 11181 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:21.115381 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2005) took 1651 ms. Trace:
I20260616 19:59:21.168311 8045 rpcz_store.cc:276] 0616 19:59:19.463548 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.114609 (+1651061us) service_pool.cc:224] Handling call
0616 19:59:21.114900 (+ 291us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:21.115088 (+ 188us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:21.115114 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:21.115153 (+ 39us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:21.115185 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:21.115236 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:21.115332 (+ 96us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:21.093822 10956 consensus_meta.cc:287] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 1.551s user 0.012s sys 0.000s
I20260616 19:59:21.000945 10995 consensus_queue.cc:237] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.043092 11196 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.170559 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:21.104418 10868 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:21.176443 8813 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:21.104030 11198 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:21.125543 7935 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:21.110765 11144 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:21.110478 11199 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:21.131472 11189 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:21.134964 8112 raft_consensus.cc:2410] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260616 19:59:21.056510 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.140866 8114 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:21.091218 11205 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:21.137638 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.119656 11168 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:21.137020 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.110456 10980 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.128304 11055 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:21.214741 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.220750 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.147172 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:21.127483 11083 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:21.114979 11133 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.163918 8084 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:21.166169 8039 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260616 19:59:21.122138 11210 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.135769 11183 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.141533 11075 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.172119 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:21.179945 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 82082.6us
I20260616 19:59:21.208461 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.210783 8110 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
W20260616 19:59:21.210776 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 85299us
I20260616 19:59:21.168210 11176 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:21.269706 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.281111 8113 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:21.248039 8112 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:21.209455 11199 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:21.176458 10868 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.207916 11171 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.284274 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:21.286438 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.247779 11055 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:21.287122 8110 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:21.214844 11079 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:21.263213 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 99383.4us
I20260616 19:59:21.281276 7934 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:21.155035 11206 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:21.283506 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.178015 10956 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.263998 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2011) took 1763 ms. Trace:
I20260616 19:59:21.283440 11189 raft_consensus.cc:2804] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:21.282132 11199 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:21.286690 8114 raft_consensus.cc:2368] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:21.261229 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.165621 11195 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.220387 11151 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:21.286192 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:21.335085 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 48980.8us
I20260616 19:59:21.178706 11071 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.296927 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.171108 11196 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.165784 11211 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.307557 8820 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:21.299506 7934 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:21.300638 11182 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:21.297386 11029 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:21.280932 11083 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:21.282136 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.218926 11157 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:21.284974 10868 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:21.303725 11082 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:21.296793 11172 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.270227 11160 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:21.306388 8039 rpcz_store.cc:276] 0616 19:59:19.500713 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.162950 (+1662237us) service_pool.cc:224] Handling call
0616 19:59:21.163169 (+ 219us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:21.263672 (+100503us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:21.263931 (+ 259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:21.344367 11072 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260616 19:59:21.309011 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:21.307826 8783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s user 0.003s sys 0.015s
I20260616 19:59:21.333943 8111 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:21.335860 7935 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:21.364924 8783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s user 0.010s sys 0.015s
I20260616 19:59:21.209601 11137 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.299036 8084 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:21.371752 8118 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 2182) took 3168 ms (client timeout 1736 ms). Trace:
W20260616 19:59:21.372347 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 73372.2us
I20260616 19:59:21.374270 8084 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:21.345552 11189 raft_consensus.cc:697] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:21.376859 8039 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:59:21.377727 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46718 (request call id 145) took 1839 ms. Trace:
I20260616 19:59:21.378123 8039 rpcz_store.cc:276] 0616 19:59:19.539060 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.375995 (+1836935us) service_pool.cc:224] Handling call
0616 19:59:21.376398 (+ 403us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:21.377386 (+ 988us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:21.377659 (+ 273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:21.348798 11197 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260616 19:59:21.344373 8110 raft_consensus.cc:2410] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260616 19:59:21.344924 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45658.7us
I20260616 19:59:21.296267 11171 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.301443 11021 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.349887 8113 raft_consensus.cc:2368] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:21.393905 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:21.395190 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.395769 8113 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:21.396629 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.352701 11029 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:21.247714 11001 consensus_queue.cc:237] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.219646 11198 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.288373 11075 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.404984 8114 raft_consensus.cc:1275] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:59:21.334944 11188 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:21.357172 11221 log.cc:927] Time spent T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.073s user 0.000s sys 0.001s
I20260616 19:59:21.335279 11078 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:21.220314 11168 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:21.361331 8037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51870 (request call id 2476) took 1859 ms (client timeout 1936 ms). Trace:
I20260616 19:59:21.390822 10923 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:59:21.413394 8037 rpcz_store.cc:269] 0616 19:59:19.502199 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:20.750422 (+1248223us) service_pool.cc:224] Handling call
0616 19:59:21.361142 (+610720us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b5400292a54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:21.361256 (+ 114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130949376}
I20260616 19:59:21.355172 11082 raft_consensus.cc:697] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:21.372231 8118 rpcz_store.cc:269] 0616 19:59:18.192489 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.192679 (+ 190us) service_pool.cc:224] Handling call
0616 19:59:21.360432 (+3167753us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400152cd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:21.360620 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3452663168}
I20260616 19:59:21.417371 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.289778 11086 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:21.364396 11072 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.397142 8113 raft_consensus.cc:2468] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
W20260616 19:59:21.401939 7934 connection.cc:256] RPC call timeout handler was delayed by -1.6695380380000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260616 19:59:21.383247 11151 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:21.386322 11157 raft_consensus.cc:2804] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:21.394467 11182 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260616 19:59:21.409857 11222 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:21.343761 11195 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.385483 11197 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:21.388821 11134 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.418095 8114 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:21.292608 11210 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.400508 11029 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:21.421228 8117 raft_consensus.cc:1275] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260616 19:59:21.405344 11021 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:21.410786 11214 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:21.414899 11158 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.473541 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1824) took 2393 ms. Trace:
I20260616 19:59:21.483320 8117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 255) took 3082 ms. Trace:
I20260616 19:59:21.443794 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.467209 8043 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:21.431311 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.465212 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:21.481930 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 60239.6us
I20260616 19:59:21.486969 8119 raft_consensus.cc:2393] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:21.483608 7969 rpcz_store.cc:276] 0616 19:59:19.080557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.451222 (+370665us) service_pool.cc:224] Handling call
0616 19:59:19.451497 (+ 275us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:20.313576 (+862079us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:20.313605 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:20.313648 (+ 43us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:20.313676 (+ 28us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:21.464638 (+1150962us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b54003f8c54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:21.464679 (+ 41us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:21.473466 (+ 8787us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1982163200}
W20260616 19:59:21.449098 11173 consensus_meta.cc:287] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.589s user 0.012s sys 0.000s
I20260616 19:59:21.493261 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.495941 8113 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:21.493649 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:21.461284 10956 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:21.481331 11157 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:21.480526 11137 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:21.502080 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.502940 8118 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:59:21.431082 11219 log.cc:927] Time spent T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.230s user 0.001s sys 0.000s
I20260616 19:59:21.438902 11138 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:21.437631 11151 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:21.429961 11072 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:21.490541 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.494328 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:21.480810 11197 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:21.475155 11133 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:21.437031 11180 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.483724 8117 rpcz_store.cc:276] 0616 19:59:18.396285 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:18.396399 (+ 114us) service_pool.cc:224] Handling call
0616 19:59:21.420710 (+3024311us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b580019f598. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:21.420909 (+ 199us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:21.478747 (+ 57838us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:21.478957 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38346496}
W20260616 19:59:21.529567 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 407996us
I20260616 19:59:21.499689 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.480026 11222 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:21.504179 11157 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:21.504951 11211 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.487784 11201 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:21.491214 11214 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:21.521283 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.565659 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.566576 8119 raft_consensus.cc:2410] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260616 19:59:21.526281 11210 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:21.526360 11195 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:21.484980 11021 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.522673 11151 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:21.522511 11138 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.552762 8113 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:21.497742 11173 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.433188 11189 consensus_queue.cc:237] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.560452 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:21.429682 11082 consensus_queue.cc:237] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:21.575708 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:22.061964 8784 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s user 0.017s sys 0.013s
W20260616 19:59:22.551877 8784 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.163s user 0.024s sys 0.032s
W20260616 19:59:22.015390 11176 consensus_meta.cc:287] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.719s user 0.010s sys 0.000s
W20260616 19:59:22.486333 7965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1816) took 3405 ms (client timeout 2053 ms). Trace:
I20260616 19:59:22.552794 11176 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:22.552904 7965 rpcz_store.cc:269] 0616 19:59:19.038627 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.038812 (+ 185us) service_pool.cc:224] Handling call
0616 19:59:22.443756 (+3404944us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54003f75d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:22.443987 (+ 231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905934208}
I20260616 19:59:22.554258 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.554697 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.554849 8008 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:22.554992 8812 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:22.555403 8811 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:21.641642 7935 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:21.923267 11055 consensus_meta.cc:287] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.628s user 0.010s sys 0.000s
I20260616 19:59:22.530165 11089 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:22.557189 11055 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.557471 11089 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:21.559043 7855 catalog_manager.cc:5226] ChangeConfig:REMOVE_PEER RPC for tablet 69c80791d0c44cb0b616bf402ae91b46 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260616 19:59:22.559144 7855 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 1.00412e+06us
I20260616 19:59:22.559482 11055 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.569000 11214 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:21.984314 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:21.622044 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:22.441425 11083 consensus_meta.cc:287] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 1.085s user 0.012s sys 0.000s
I20260616 19:59:22.561643 8037 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:22.019306 11138 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.561895 11083 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.562933 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:21.573637 11196 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:21.751838 11191 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.081s user 0.018s sys 0.000s
I20260616 19:59:21.811890 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:21.577976 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:21.665498 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:21.953361 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:21.823777 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 336773us
W20260616 19:59:22.092375 7934 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.531s user 0.009s sys 0.005s
I20260616 19:59:21.723248 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.147538 8042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2008) took 2683 ms. Trace:
I20260616 19:59:21.952900 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:21.662441 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:21.606511 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:21.594456 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.556458 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:22.564079 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:22.571097 8042 rpcz_store.cc:276] 0616 19:59:19.464289 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.144985 (+1680696us) service_pool.cc:224] Handling call
0616 19:59:21.154566 (+ 9581us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:21.356098 (+201532us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:21.356139 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:21.356207 (+ 68us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:21.356239 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:22.147195 (+790956us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54000e79d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:22.147268 (+ 73us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:22.147458 (+ 190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462997376}
I20260616 19:59:22.602473 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:22.603328 8042 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:22.571816 7961 raft_consensus.cc:2393] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:22.575135 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1932) took 1017 ms. Trace:
W20260616 19:59:22.576864 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 935259us
I20260616 19:59:22.573993 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:22.607244 7935 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:21.795931 8011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s user 0.012s sys 0.023s
W20260616 19:59:22.608806 8011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.464s user 0.014s sys 0.028s
I20260616 19:59:22.605726 8114 rpcz_store.cc:276] 0616 19:59:21.557698 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.572104 (+ 14406us) service_pool.cc:224] Handling call
0616 19:59:22.575055 (+1002951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:22.605656 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:22.570364 7934 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.010s user 0.009s sys 0.010s
I20260616 19:59:22.585745 7965 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:22.610769 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 1.05016e+06us
I20260616 19:59:22.608218 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:22.612069 8043 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:22.612771 8117 raft_consensus.cc:2393] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:22.612927 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:22.613497 8043 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:22.599974 8037 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:21.985172 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:22.614281 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:22.615270 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:22.615810 8043 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:22.615298 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:22.645900 8113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 2203) took 1158 ms. Trace:
I20260616 19:59:22.647459 8113 rpcz_store.cc:276] 0616 19:59:21.393838 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.498722 (+104884us) service_pool.cc:224] Handling call
0616 19:59:22.551838 (+1053116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:22.646358 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:22.646334 8119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1931) took 1060 ms. Trace:
I20260616 19:59:22.648721 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.649767 8113 raft_consensus.cc:2393] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 12: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:59:22.649914 8040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 2007) took 3149 ms (client timeout 1810 ms). Trace:
I20260616 19:59:22.617913 8112 raft_consensus.cc:2410] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260616 19:59:22.650590 8040 rpcz_store.cc:269] 0616 19:59:19.464005 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:20.239159 (+775154us) service_pool.cc:224] Handling call
0616 19:59:22.613674 (+2374515us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54003aaa54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:22.613764 (+ 90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857480448}
I20260616 19:59:22.652206 8117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 2212) took 1086 ms. Trace:
W20260616 19:59:22.440960 11029 consensus_meta.cc:287] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.953s user 0.011s sys 0.000s
I20260616 19:59:22.644470 8042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 2035) took 1021 ms. Trace:
I20260616 19:59:22.652956 11029 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.652900 8117 rpcz_store.cc:276] 0616 19:59:21.527293 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.531130 (+ 3837us) service_pool.cc:224] Handling call
0616 19:59:22.613146 (+1082016us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:22.653262 8042 rpcz_store.cc:276] 0616 19:59:21.622952 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:22.604138 (+981186us) service_pool.cc:224] Handling call
0616 19:59:22.644365 (+ 40227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:22.616896 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 1.56385e+06us
I20260616 19:59:22.655200 11029 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:21.855854 11136 consensus_meta.cc:287] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 1.058s user 0.007s sys 0.004s
I20260616 19:59:22.656507 11136 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.658104 8044 raft_consensus.cc:1240] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260616 19:59:22.658623 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:22.659293 7961 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
W20260616 19:59:21.584942 11077 consensus_meta.cc:287] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.709s user 0.011s sys 0.001s
I20260616 19:59:22.648955 8119 rpcz_store.cc:276] 0616 19:59:21.557366 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.567317 (+ 9951us) service_pool.cc:224] Handling call
0616 19:59:22.617531 (+1050214us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:22.660585 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 53483.9us
I20260616 19:59:22.664263 7935 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:22.665176 8038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46718 (request call id 144) took 3261 ms. Trace:
I20260616 19:59:22.665795 8038 rpcz_store.cc:276] 0616 19:59:19.403929 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.453403 (+ 49474us) service_pool.cc:224] Handling call
0616 19:59:19.453796 (+ 393us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:20.711136 (+1257340us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:22.664929 (+1953793us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54000e7ed4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:22.665112 (+ 183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957836544}
I20260616 19:59:21.593501 11171 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:22.666945 8112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1933) took 1109 ms. Trace:
W20260616 19:59:21.688320 11147 consensus_meta.cc:287] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.905s user 0.012s sys 0.000s
I20260616 19:59:22.559834 11089 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.095428 7865 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 6 to 7, leader changed from c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) to de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130). New cstate: current_term: 7 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260616 19:59:22.014587 11199 consensus_meta.cc:287] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.678s user 0.005s sys 0.005s
I20260616 19:59:21.564719 11075 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:22.670832 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:22.063365 10868 consensus_meta.cc:287] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.700s user 0.010s sys 0.002s
I20260616 19:59:21.760365 10956 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
W20260616 19:59:22.565011 11191 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.894s user 0.027s sys 0.029s
I20260616 19:59:22.667817 8112 rpcz_store.cc:276] 0616 19:59:21.558018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.722922 (+164904us) service_pool.cc:224] Handling call
0616 19:59:22.666887 (+943965us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:22.680008 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:22.055908 11145 raft_consensus.cc:3055] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
W20260616 19:59:22.624855 10995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.135s user 0.023s sys 0.019s
I20260616 19:59:21.751577 11168 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.294395 11170 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:21.765043 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.729983 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:21.950851 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.646245 11138 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:21.834576 11222 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:22.649842 11150 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:22.618677 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.017649 11021 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:21.804311 11092 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.877s
I20260616 19:59:22.714293 11145 raft_consensus.cc:740] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:21.585103 11201 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.711597 7961 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:22.055686 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.564384 11083 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:22.666131 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.726123 7965 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:22.726393 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.727468 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.673573 11171 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:22.019127 11072 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:22.711717 11075 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:22.735810 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:22.686436 11089 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:22.738040 8819 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260616 19:59:22.739226 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 180985us
W20260616 19:59:22.727603 10995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.238s user 0.026s sys 0.026s
I20260616 19:59:22.669081 11077 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:22.688068 11199 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.735685 8813 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52474 (request call id 244) took 1001 ms. Trace:
I20260616 19:59:22.680173 11147 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.742430 11232 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:22.729878 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.687446 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.715996 10868 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:22.744726 11138 raft_consensus.cc:697] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:22.754628 11157 consensus_meta.cc:287] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 1.186s user 0.013s sys 0.000s
I20260616 19:59:22.730226 10956 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:22.758531 11136 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.825425 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:22.803484 8008 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:22.808883 8009 leader_election.cc:400] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
I20260616 19:59:22.807646 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:22.826699 8009 leader_election.cc:403] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260616 19:59:22.736308 11170 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.841938 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.802511 11171 raft_consensus.cc:697] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:22.868105 8118 raft_consensus.cc:2393] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:22.868254 8008 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:22.868997 7963 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:22.832086 7970 raft_consensus.cc:2393] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:22.870394 8119 raft_consensus.cc:2410] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260616 19:59:22.867967 8114 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:22.807543 11232 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:22.869655 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 64756.2us
W20260616 19:59:22.890146 11237 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.143s user 0.000s sys 0.000s
I20260616 19:59:22.890595 8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52474 (request call id 245) took 1132 ms. Trace:
I20260616 19:59:22.802894 8813 rpcz_store.cc:276] 0616 19:59:21.734226 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.752330 (+ 18104us) service_pool.cc:224] Handling call
0616 19:59:22.735624 (+983294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:22.824985 11157 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.884763 8112 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 10: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:22.855690 11070 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:22.797245 11075 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.908038 11075 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.900362 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.873682 11018 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.890662 11072 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:22.872041 11243 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:22.871239 11155 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:22.929188 11147 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:22.929360 11155 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:22.929798 11155 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:22.930619 11108 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.892076 10868 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:22.903540 8815 rpcz_store.cc:276] 0616 19:59:21.756674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.760311 (+ 3637us) service_pool.cc:224] Handling call
0616 19:59:22.884126 (+1123815us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:22.870886 10959 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.935760 11165 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.935907 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.936762 8119 raft_consensus.cc:2393] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:59:22.937139 11151 consensus_meta.cc:287] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 1.249s user 0.011s sys 0.000s
I20260616 19:59:22.937734 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:22.909235 11214 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:22.940696 8116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2196) took 3871 ms. Trace:
I20260616 19:59:22.940979 7968 raft_consensus.cc:1240] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:22.927644 11171 consensus_queue.cc:237] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.941143 8116 rpcz_store.cc:276] 0616 19:59:19.069592 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.069735 (+ 143us) service_pool.cc:224] Handling call
0616 19:59:19.070048 (+ 313us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:20.791216 (+1721168us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:22.940446 (+2149230us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54002cb854. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:22.940633 (+ 187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495544064}
I20260616 19:59:22.942196 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 326) took 1458 ms. Trace:
I20260616 19:59:22.942503 8116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "001495805f9843f0aac7937a5905dc82"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:22.942535 7968 rpcz_store.cc:276] 0616 19:59:21.484038 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.484292 (+ 254us) service_pool.cc:224] Handling call
0616 19:59:21.484585 (+ 293us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:22.941485 (+1456900us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:22.941984 (+ 499us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003bced4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:22.942145 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3348933888}
I20260616 19:59:22.943269 8116 raft_consensus.cc:606] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:22.943681 8116 raft_consensus.cc:614] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:22.944026 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:22.944855 7968 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:22.945626 8119 raft_consensus.cc:2368] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 11. Current term is 12.
I20260616 19:59:22.946115 8111 raft_consensus.cc:1240] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:22.947446 11142 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.877535 11170 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.869606 11168 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
W20260616 19:59:22.928263 8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52458 (request call id 360) took 1781 ms (client timeout 1928 ms). Trace:
W20260616 19:59:22.996001 8820 rpcz_store.cc:269] 0616 19:59:21.146712 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.146950 (+ 238us) service_pool.cc:224] Handling call
0616 19:59:22.928017 (+1781067us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b5400462654. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:22.928184 (+ 167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135711104}
I20260616 19:59:22.928491 10956 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:22.928871 11077 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:22.928300 11190 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.937520 11151 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:22.874715 11202 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:22.976783 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 66247.2us
I20260616 19:59:22.986532 7968 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:23.015033 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2512) took 1086 ms. Trace:
W20260616 19:59:22.911680 11197 consensus_meta.cc:287] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 1.373s user 0.009s sys 0.002s
I20260616 19:59:23.021445 11139 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:23.011229 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 65684.3us
W20260616 19:59:23.011762 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 354161us
I20260616 19:59:22.902899 11232 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:22.910528 11176 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:22.911907 11157 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:22.910037 11173 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:22.910251 11070 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:23.049186 11070 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:22.994436 7963 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:23.016676 8009 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:23.052496 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2520) took 1121 ms. Trace:
I20260616 19:59:23.052881 7963 rpcz_store.cc:276] 0616 19:59:21.930678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:22.807381 (+876703us) service_pool.cc:224] Handling call
0616 19:59:23.052424 (+245043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.053270 8009 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.054026 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:22.901293 11145 consensus_queue.cc:260] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.010197 11138 consensus_queue.cc:237] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.027798 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.059487 7961 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:23.028702 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:23.029436 7967 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.986s user 0.001s sys 0.009s
W20260616 19:59:23.009338 8041 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.965s user 0.000s sys 0.008s
W20260616 19:59:23.060957 8041 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.016s user 0.000s sys 0.008s
I20260616 19:59:23.016939 7968 rpcz_store.cc:276] 0616 19:59:21.928495 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:22.943775 (+1015280us) service_pool.cc:224] Handling call
0616 19:59:23.014948 (+ 71173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.062319 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:23.063037 7968 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:23.060504 7965 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:23.069373 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2523) took 1128 ms. Trace:
I20260616 19:59:23.069981 7961 rpcz_store.cc:276] 0616 19:59:21.931477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.019257 (+1087780us) service_pool.cc:224] Handling call
0616 19:59:23.060087 (+ 40830us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.071334 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.038882 8011 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:23.008989 11201 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:23.029258 11229 log.cc:927] Time spent T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.140s user 0.000s sys 0.001s
I20260616 19:59:23.054458 8009 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:23.060477 7967 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.017s user 0.001s sys 0.009s
I20260616 19:59:23.077275 11142 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:23.078985 11190 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:23.080618 11190 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:23.080976 11190 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:23.080015 8009 leader_election.cc:400] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:23.084643 8009 leader_election.cc:403] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:59:23.085296 11068 raft_consensus.cc:3060] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 8
I20260616 19:59:23.085958 8009 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.087695 11240 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:23.088165 11240 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:23.090420 11209 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:59:23.088567 11240 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:23.088219 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.096427 7961 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:23.078691 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 328) took 1594 ms. Trace:
I20260616 19:59:23.101406 7967 rpcz_store.cc:276] 0616 19:59:21.484642 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.484742 (+ 100us) service_pool.cc:224] Handling call
0616 19:59:21.489780 (+ 5038us) raft_consensus.cc:1471] Updating replica for 2 ops
0616 19:59:21.528547 (+ 38767us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:21.528575 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0616 19:59:21.529256 (+ 681us) log.cc:844] Serialized 52 byte log entry
0616 19:59:21.951966 (+422710us) raft_consensus.cc:1645] Marking committed up to 11
0616 19:59:23.076225 (+1124259us) log.cc:844] Serialized 14 byte log entry
0616 19:59:23.077443 (+ 1218us) log.cc:844] Serialized 14 byte log entry
0616 19:59:23.078366 (+ 923us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.078435 (+ 69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:23.078475 (+ 40us) raft_consensus.cc:1685] finished
0616 19:59:23.078486 (+ 11us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:23.078632 (+ 146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38377,"raft.queue_time_us":59852,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":6869,"thread_start_us":1415368,"threads_started":2,"wal-append.queue_time_us":1158238}
I20260616 19:59:23.103600 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.104329 7967 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
W20260616 19:59:23.090952 8009 connection.cc:256] RPC call timeout handler was delayed by -1.3034390780001104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2646s for a response
I20260616 19:59:23.108069 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.108299 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.107782 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:23.164908 7961 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:23.111302 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.168467 7967 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:23.169108 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2524) took 1132 ms. Trace:
I20260616 19:59:23.110169 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:23.109558 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.112025 11190 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.032771 10956 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:23.169831 7934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s user 0.022s sys 0.064s
W20260616 19:59:23.174495 8011 connection.cc:256] RPC call timeout handler was delayed by -4.7474092390000351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260616 19:59:23.174711 7934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s user 0.022s sys 0.065s
W20260616 19:59:23.175004 8011 connection.cc:256] RPC call timeout handler was delayed by -4.5143092389999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260616 19:59:23.169520 7967 rpcz_store.cc:276] 0616 19:59:22.036359 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.110994 (+1074635us) service_pool.cc:224] Handling call
0616 19:59:23.169003 (+ 58009us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.159224 8038 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:23.171527 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2521) took 1240 ms. Trace:
I20260616 19:59:23.171126 8040 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:23.176770 7963 rpcz_store.cc:276] 0616 19:59:21.930952 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.053708 (+1122756us) service_pool.cc:224] Handling call
0616 19:59:23.171466 (+117758us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.058493 11161 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:23.025182 11197 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.200222 11161 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:23.222743 11161 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:23.170339 11173 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:23.202004 11144 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:23.203894 11198 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.174360 11114 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:23.211730 8044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2004) took 3748 ms. Trace:
I20260616 19:59:23.237301 8044 rpcz_store.cc:276] 0616 19:59:19.463263 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.110814 (+1647551us) service_pool.cc:224] Handling call
0616 19:59:21.111224 (+ 410us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:22.663612 (+1552388us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.211416 (+547804us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54000e8dd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:23.211657 (+ 241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3556821888}
I20260616 19:59:23.212457 11152 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:23.187871 11171 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:23.171113 11199 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.197392 11145 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:23.244329 11129 raft_consensus.cc:3055] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:23.244757 11129 raft_consensus.cc:740] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:23.199180 11139 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:23.232405 11197 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.188526 10956 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.228106 11173 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:23.243494 11151 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.244488 11171 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.232527 11144 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:23.250867 11173 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:23.259310 10956 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:23.254329 11129 consensus_queue.cc:260] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.260753 10969 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:23.261811 11129 raft_consensus.cc:3060] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:23.261930 10980 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.255383 11214 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:23.248925 11139 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.271780 11190 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:23.314428 7935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s user 0.011s sys 0.017s
W20260616 19:59:23.315434 7935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s user 0.011s sys 0.017s
I20260616 19:59:23.315999 7964 raft_consensus.cc:2368] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12.
I20260616 19:59:23.175206 7965 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
W20260616 19:59:23.317526 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 653293us
I20260616 19:59:23.318027 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2516) took 1388 ms. Trace:
I20260616 19:59:23.318552 7965 rpcz_store.cc:276] 0616 19:59:21.929578 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.019790 (+1090212us) service_pool.cc:224] Handling call
0616 19:59:23.317779 (+297989us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.321677 7967 raft_consensus.cc:1240] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260616 19:59:23.322631 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2514) took 1393 ms. Trace:
I20260616 19:59:23.322798 7935 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.323055 7967 rpcz_store.cc:276] 0616 19:59:21.929039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.320223 (+1391184us) service_pool.cc:224] Handling call
0616 19:59:23.321052 (+ 829us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:23.322257 (+ 1205us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.322533 (+ 276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.324828 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.326517 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:23.330430 7935 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.336746 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.207118 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2513) took 1278 ms. Trace:
I20260616 19:59:23.340518 7969 rpcz_store.cc:276] 0616 19:59:21.928763 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.206242 (+1277479us) service_pool.cc:224] Handling call
0616 19:59:23.206569 (+ 327us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:23.206759 (+ 190us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:23.206789 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:23.206829 (+ 40us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:23.206878 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.206936 (+ 58us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:23.207053 (+ 117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:23.337493 8011 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.794s (SENT) [suppressed 13 similar messages]
I20260616 19:59:23.331139 7968 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:23.217048 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2039) took 1593 ms. Trace:
W20260616 19:59:23.348412 8085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s user 0.000s sys 0.008s
W20260616 19:59:23.348866 8085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s user 0.000s sys 0.008s
W20260616 19:59:23.349532 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 684904us
W20260616 19:59:23.344581 8084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s user 0.043s sys 0.094s
I20260616 19:59:23.347479 8043 rpcz_store.cc:276] 0616 19:59:21.624008 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:22.616330 (+992322us) service_pool.cc:224] Handling call
0616 19:59:22.616621 (+ 291us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:22.757896 (+141275us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:22.757925 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:22.806340 (+ 48415us) log.cc:844] Serialized 26 byte log entry
0616 19:59:23.171327 (+364987us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:59:23.171363 (+ 36us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.171438 (+ 75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:23.216737 (+ 45299us) raft_consensus.cc:1685] finished
0616 19:59:23.216757 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:23.216931 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128209,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":10890,"thread_start_us":419322,"threads_started":2,"wal-append.queue_time_us":387994}
I20260616 19:59:23.360464 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2525) took 1307 ms. Trace:
W20260616 19:59:23.363723 8084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s user 0.044s sys 0.095s
I20260616 19:59:23.341122 7935 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.366227 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2517) took 1436 ms. Trace:
I20260616 19:59:23.366839 7968 rpcz_store.cc:276] 0616 19:59:21.929851 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.061921 (+1132070us) service_pool.cc:224] Handling call
0616 19:59:23.362904 (+300983us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.366510 11080 consensus_queue.cc:1059] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:59:23.368386 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:23.369261 11080 raft_consensus.cc:3055] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:23.369786 11080 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:23.369706 11155 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.370875 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.372820 10885 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:23.374359 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.373983 10885 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.375428 8044 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:23.172055 11245 consensus_queue.cc:1048] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.321s
I20260616 19:59:23.382822 11139 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:23.396600 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 53419.2us
W20260616 19:59:23.399705 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 829403us
I20260616 19:59:23.409456 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:23.400928 11080 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.420385 11158 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260616 19:59:23.420576 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 52507.8us
I20260616 19:59:23.421072 11158 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:23.421602 11158 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:23.421681 11263 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:23.422312 11263 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:23.422627 8085 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:23.423903 11258 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:23.424441 11258 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:23.424616 11162 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:23.424710 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.425269 11162 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:23.425853 11162 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:23.421072 11080 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:23.426549 11152 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.422775 11263 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:23.404572 11089 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:23.423897 8085 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:23.399607 11171 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:23.430354 11258 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:23.440009 11147 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:23.440598 11147 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:23.441061 11147 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:23.399559 11068 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:23.413451 11112 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260616 19:59:23.447438 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 106628us
I20260616 19:59:23.452409 11139 raft_consensus.cc:2764] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260616 19:59:23.455458 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 46171.6us
I20260616 19:59:23.431540 11090 raft_consensus.cc:2764] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260616 19:59:23.465191 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.467905 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:23.400211 11021 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260616 19:59:23.504321 7968 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260616 19:59:23.505668 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:23.505690 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.508060 7969 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:23.513762 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 90138.5us
I20260616 19:59:23.453524 11152 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.471905 11089 raft_consensus.cc:697] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:23.523591 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 63985.9us
W20260616 19:59:23.526710 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 58936.7us
I20260616 19:59:23.546005 11240 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.548207 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2507) took 1621 ms. Trace:
I20260616 19:59:23.548496 10969 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:23.549125 11240 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.550431 10885 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.551435 11155 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:23.551108 11089 consensus_queue.cc:237] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:23.552923 7966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1815) took 4519 ms (client timeout 1794 ms). Trace:
W20260616 19:59:23.553371 7966 rpcz_store.cc:269] 0616 19:59:19.032994 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.033142 (+ 148us) service_pool.cc:224] Handling call
0616 19:59:23.552417 (+4519275us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003f7854. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.552699 (+ 282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2806631168}
I20260616 19:59:23.549204 10969 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:23.548748 7970 rpcz_store.cc:276] 0616 19:59:21.926800 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.926924 (+ 124us) service_pool.cc:224] Handling call
0616 19:59:23.547749 (+1620825us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54001db354. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.547979 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610035072}
I20260616 19:59:23.558899 8084 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:23.562367 11015 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:23.562906 11015 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:23.563329 11015 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:23.563539 10998 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:23.564280 10998 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:23.564785 10998 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:23.563700 8041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2010) took 4098 ms. Trace:
I20260616 19:59:23.566573 8041 rpcz_store.cc:276] 0616 19:59:19.464787 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.146176 (+1681389us) service_pool.cc:224] Handling call
0616 19:59:21.146491 (+ 315us) raft_consensus.cc:1471] Updating replica for 2 ops
0616 19:59:21.304293 (+157802us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:21.304323 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0616 19:59:21.304898 (+ 575us) log.cc:844] Serialized 52 byte log entry
0616 19:59:21.682334 (+377436us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:23.555418 (+1873084us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b18003c86e0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86ee8c85c 00007fe86c216c0b 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c230a95 00007fe872cdf247 00007fe86f2efd87 00007fe86f2ff49f
0616 19:59:23.555667 (+ 249us) log.cc:844] Serialized 14 byte log entry
0616 19:59:23.562818 (+ 7151us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.562963 (+ 145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:23.563046 (+ 83us) raft_consensus.cc:1685] finished
0616 19:59:23.563194 (+ 148us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:23.563468 (+ 274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38813,"prepare.queue_time_us":1097304,"prepare.run_cpu_time_us":533,"prepare.run_wall_time_us":8247,"raft.queue_time_us":142295,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":11510,"spinlock_wait_cycles":252101504,"thread_start_us":1497126,"threads_started":3,"wal-append.queue_time_us":1424427}
I20260616 19:59:23.555985 10969 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.580498 7935 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:23.581097 11266 log.cc:927] Time spent T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260616 19:59:23.584838 11021 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260616 19:59:23.585366 11021 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:23.585670 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 60664.4us
I20260616 19:59:23.586696 8085 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:23.585824 11021 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:23.588135 8085 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:23.591544 7961 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
W20260616 19:59:23.593029 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:23.593632 8113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2216) took 1037 ms. Trace:
I20260616 19:59:23.620496 8113 rpcz_store.cc:276] 0616 19:59:22.556013 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:22.658900 (+102887us) service_pool.cc:224] Handling call
0616 19:59:22.659308 (+ 408us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:23.036662 (+377354us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:23.036700 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:23.037180 (+ 480us) log.cc:844] Serialized 26 byte log entry
0616 19:59:23.188002 (+150822us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:23.188047 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.188122 (+ 75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:23.593150 (+405028us) raft_consensus.cc:1685] finished
0616 19:59:23.593183 (+ 33us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:23.593392 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9084,"raft.queue_time_us":98173,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":11349,"thread_start_us":511837,"threads_started":2,"wal-append.queue_time_us":347846}
I20260616 19:59:23.624778 7970 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:23.625070 8085 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:23.625718 10956 raft_consensus.cc:2804] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:23.626807 8085 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:23.630456 11173 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.639223 11015 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.588254 11246 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:23.646389 7969 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:23.658607 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2515) took 1711 ms. Trace:
I20260616 19:59:23.675524 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.675848 7961 rpcz_store.cc:276] 0616 19:59:21.929303 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.105094 (+1175791us) service_pool.cc:224] Handling call
0616 19:59:23.620775 (+515681us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:23.679757 11246 raft_consensus.cc:697] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:59:23.711797 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:23.710541 8045 raft_consensus.cc:1240] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
W20260616 19:59:23.690588 11078 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.799s user 0.016s sys 0.022s
W20260616 19:59:23.770462 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 108466us
W20260616 19:59:23.707485 8118 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1930) took 2150 ms (client timeout 1983 ms). Trace:
W20260616 19:59:23.709772 8036 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 2003) took 4248 ms (client timeout 1538 ms). Trace:
W20260616 19:59:23.699105 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 46776.3us
I20260616 19:59:23.702680 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1936) took 1039 ms. Trace:
I20260616 19:59:23.723654 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.736577 8115 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
W20260616 19:59:23.789484 8118 rpcz_store.cc:269] 0616 19:59:21.557083 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.557385 (+ 302us) service_pool.cc:224] Handling call
0616 19:59:23.707050 (+2149665us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400056754. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.707275 (+ 225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677828992}
W20260616 19:59:23.789855 8036 rpcz_store.cc:269] 0616 19:59:19.461675 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:19.462158 (+ 483us) service_pool.cc:224] Handling call
0616 19:59:23.709354 (+4247196us) spinlock_profiling.cc:243] Waited 91.5 ms on lock 0x7b54003ac0d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.709567 (+ 213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210508160}
W20260616 19:59:23.797158 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:23.800835 8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52474 (request call id 243) took 2082 ms. Trace:
I20260616 19:59:23.802958 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.804109 7961 raft_consensus.cc:1275] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:59:23.363520 7967 rpcz_store.cc:276] 0616 19:59:22.050798 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.324667 (+1273869us) service_pool.cc:224] Handling call
0616 19:59:23.325241 (+ 574us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:23.325812 (+ 571us) raft_consensus.cc:1534] Early marking committed up to index 4
0616 19:59:23.326499 (+ 687us) log.cc:844] Serialized 14 byte log entry
0616 19:59:23.357867 (+ 31368us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:23.357922 (+ 55us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:23.357971 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.358030 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:23.358160 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31194,"thread_start_us":6163,"threads_started":1}
I20260616 19:59:23.720659 11157 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:23.811728 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.812534 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.814147 8820 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:23.814810 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:23.815836 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.708379 11218 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:23.817032 11218 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.818297 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.819154 8040 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:59:23.820601 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.821564 8044 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 6: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:23.821594 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.705152 11145 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:23.822810 11145 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.823539 8785 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:23.824294 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.825169 11145 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:23.825564 11206 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:23.826052 11206 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:23.826460 11206 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:23.826800 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.829833 11138 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:23.830560 11138 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.830847 10885 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:23.831584 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2475) took 4330 ms. Trace:
W20260616 19:59:23.789423 11078 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.062s user 0.021s sys 0.025s
W20260616 19:59:23.823683 8115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1920) took 2717 ms (client timeout 2037 ms). Trace:
W20260616 19:59:23.833657 8115 rpcz_store.cc:269] 0616 19:59:21.105871 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.137470 (+ 31599us) service_pool.cc:224] Handling call
0616 19:59:23.823482 (+2686012us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b5400102f54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.823619 (+ 137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5807527808}
I20260616 19:59:23.679163 10956 raft_consensus.cc:697] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:59:23.722064 11070 consensus_meta.cc:287] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.672s user 0.013s sys 0.005s
I20260616 19:59:23.705926 11068 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:23.849192 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:23.849102 11068 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.832050 8045 rpcz_store.cc:276] 0616 19:59:19.501045 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.169335 (+1668290us) service_pool.cc:224] Handling call
0616 19:59:23.710032 (+2540697us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:23.793514 (+ 83482us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.831390 (+ 37876us) spinlock_profiling.cc:243] Waited 2.54 s on lock 0x7b54003ac0d0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.831521 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5841962368}
I20260616 19:59:23.833719 11157 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:23.851039 11157 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:23.851454 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:23.851886 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.852119 8045 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 8
I20260616 19:59:23.852530 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:23.853295 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:23.856287 11243 consensus_meta.cc:287] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.681s user 0.006s sys 0.006s
I20260616 19:59:23.856889 11243 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7.
I20260616 19:59:23.857921 8036 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:23.858366 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.858141 11070 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:23.840857 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.843016 8118 raft_consensus.cc:1240] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:23.859648 7968 raft_consensus.cc:2368] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:23.861092 8042 raft_consensus.cc:2393] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:23.845963 7964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2511) took 1917 ms (client timeout 1569 ms). Trace:
I20260616 19:59:23.842859 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
W20260616 19:59:23.861678 7964 rpcz_store.cc:269] 0616 19:59:21.928214 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:22.569929 (+641715us) service_pool.cc:224] Handling call
0616 19:59:23.845717 (+1275788us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54003b0e54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.845889 (+ 172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532900224}
I20260616 19:59:23.843947 11179 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260616 19:59:23.846393 8113 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:23.847213 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.846638 11148 consensus_queue.cc:1048] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:23.723956 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:23.863607 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:23.865226 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:23.866093 8044 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:23.865257 8041 raft_consensus.cc:1240] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260616 19:59:23.863751 8816 raft_consensus.cc:1813] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Handling vote request from an unknown peer 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:23.868171 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.869504 7968 raft_consensus.cc:1275] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:59:23.870787 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.872008 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.873245 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.872839 11246 consensus_queue.cc:237] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.726140 11147 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.722723 11015 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.874402 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.875630 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.875985 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.876814 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.702417 11191 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.178s
I20260616 19:59:23.698968 11055 consensus_queue.cc:1059] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.206s
I20260616 19:59:23.880941 10998 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.801553 8817 rpcz_store.cc:276] 0616 19:59:21.718520 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.718647 (+ 127us) service_pool.cc:224] Handling call
0616 19:59:22.932071 (+1213424us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:22.932503 (+ 432us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:22.932530 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:22.932876 (+ 346us) log.cc:844] Serialized 26 byte log entry
0616 19:59:23.057204 (+124328us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:23.057341 (+ 137us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:23.057543 (+ 202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:23.433743 (+376200us) raft_consensus.cc:1685] finished
0616 19:59:23.433783 (+ 40us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:23.800321 (+366538us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400462650. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.800609 (+ 288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":2789686016,"thread_start_us":121232,"threads_started":1,"wal-append.queue_time_us":465319}
I20260616 19:59:23.857940 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:23.882154 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:23.884543 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 257943us
I20260616 19:59:23.842768 11162 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.861052 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:23.819985 11218 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:23.737953 11263 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.723886 11173 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:23.790480 8114 rpcz_store.cc:276] 0616 19:59:22.663548 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:22.689935 (+ 26387us) service_pool.cc:224] Handling call
0616 19:59:23.702151 (+1012216us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400437954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:23.702408 (+ 257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208558336}
I20260616 19:59:23.729452 11257 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260616 19:59:23.935379 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 49186.4us
I20260616 19:59:23.942546 10956 consensus_queue.cc:237] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.944756 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:23.946171 7963 raft_consensus.cc:2368] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:23.946455 8117 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260616 19:59:24.012645 7965 raft_consensus.cc:2393] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.011549 7961 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:23.949117 8045 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:23.960210 8113 raft_consensus.cc:1240] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:24.011516 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.955802 8044 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 12.
W20260616 19:59:24.007586 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 59628.9us
I20260616 19:59:24.020444 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:23.946652 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.030059 8816 raft_consensus.cc:2410] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260616 19:59:24.014088 11021 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:23.930620 11258 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.031951 11021 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:24.032445 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.032951 7964 raft_consensus.cc:2393] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.033761 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.959666 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:24.034608 8112 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:23.959450 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:24.035118 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:23.960192 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.036250 8110 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 10: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:24.037556 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.038225 7963 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:24.039219 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.039569 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.040364 7965 raft_consensus.cc:2393] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.033851 11162 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:24.036285 8036 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:24.049968 7968 raft_consensus.cc:2368] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 11. Current term is 12.
I20260616 19:59:23.958943 8085 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:24.057389 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.062214 7969 raft_consensus.cc:2368] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:24.068203 8038 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260616 19:59:24.068203 8040 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:24.070191 8112 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:24.075205 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.082232 8116 raft_consensus.cc:2393] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:24.082899 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.083137 11263 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.083904 8119 raft_consensus.cc:2393] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:24.084833 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.024019 11257 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:23.959306 11210 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:24.086485 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 130723us
I20260616 19:59:24.131932 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:24.090185 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:24.312067 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.268127 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:24.098937 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.108358 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.318584 7966 raft_consensus.cc:1240] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260616 19:59:24.317732 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:24.131021 7963 raft_consensus.cc:2393] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 12: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.170594 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:24.086530 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.209059 7969 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:59:24.338692 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.346269 7970 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.337136 8041 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 12.
I20260616 19:59:24.276284 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:24.179315 11161 consensus_meta.cc:287] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.733s user 0.009s sys 0.005s
W20260616 19:59:24.179527 11232 consensus_meta.cc:287] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.985s user 0.008s sys 0.008s
I20260616 19:59:24.354817 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:24.345583 11158 consensus_meta.cc:287] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.923s user 0.016s sys 0.016s
I20260616 19:59:24.359673 8114 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:24.264289 11138 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:24.339033 11257 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:24.205094 10998 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.365132 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:24.368516 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:24.149647 11145 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:24.384591 11145 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:24.375087 11158 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.376046 11161 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.389623 8037 raft_consensus.cc:1240] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260616 19:59:24.390698 11161 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:24.378844 11232 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.375492 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.377053 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:24.377791 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.379105 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.396770 7969 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.379558 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:24.383405 8119 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:24.380726 8110 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:24.385196 11145 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:24.191684 11157 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.363188 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.393033 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.384488 8113 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:24.394450 7966 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9. Ops: []
W20260616 19:59:24.394989 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:24.387786 11158 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:24.405857 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.406600 8042 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:24.407621 7966 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:59:24.408685 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.408823 8118 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 3.
I20260616 19:59:24.409461 8044 raft_consensus.cc:2368] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12.
I20260616 19:59:24.410001 7969 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:24.410316 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.411062 8044 raft_consensus.cc:2393] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 6: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:24.411799 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.412441 8044 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 8
I20260616 19:59:24.416841 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.416832 8011 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:24.419755 8044 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:24.406534 7963 raft_consensus.cc:1240] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260616 19:59:24.428071 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.274961 11173 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:24.368705 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:24.429039 7965 raft_consensus.cc:2410] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260616 19:59:24.435905 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.438000 8036 raft_consensus.cc:2393] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:24.438092 11152 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:24.438390 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.439294 11068 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:24.448400 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.435993 11078 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:59:24.452049 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:24.454216 8119 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 10: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:24.460315 8118 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
W20260616 19:59:24.476642 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 60089.2us
I20260616 19:59:24.488749 7962 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:24.489327 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.459630 8042 raft_consensus.cc:2410] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260616 19:59:24.481698 11015 raft_consensus.cc:3055] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
W20260616 19:59:24.496922 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 66822.8us
I20260616 19:59:24.460196 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.458483 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.465121 7970 raft_consensus.cc:2393] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.468226 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.497076 11015 raft_consensus.cc:740] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:24.486853 8037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46718 (request call id 154) took 1377 ms. Trace:
I20260616 19:59:24.491711 8113 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:24.505357 8112 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:24.506902 8037 rpcz_store.cc:276] 0616 19:59:23.109782 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.346203 (+236421us) service_pool.cc:224] Handling call
0616 19:59:23.346967 (+ 764us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:24.389994 (+1043027us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:24.486469 (+ 96475us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000e7ed4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:24.486741 (+ 272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397099776}
I20260616 19:59:24.514567 8110 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:24.349354 11210 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.517418 8118 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 10. Current term is 11. Ops: []
W20260616 19:59:24.471554 11144 consensus_meta.cc:287] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 1.206s user 0.007s sys 0.005s
I20260616 19:59:24.467438 11232 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:24.465194 11157 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:24.523012 8117 raft_consensus.cc:1240] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:24.524312 11161 consensus_queue.cc:1048] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:24.530604 11257 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.519120 10969 consensus_queue.cc:1059] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260616 19:59:24.532951 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:24.533540 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.533746 8036 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:24.534209 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.536439 8038 raft_consensus.cc:2468] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:24.537755 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.533164 10969 raft_consensus.cc:3055] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:24.542476 8116 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:24.543881 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.545171 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:24.547268 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:24.547497 8111 raft_consensus.cc:3060] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:24.549511 8036 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
W20260616 19:59:24.540697 11206 consensus_meta.cc:287] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.709s user 0.013s sys 0.000s
I20260616 19:59:24.553203 8110 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:24.554647 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.552455 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.489367 11078 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.557008 8037 raft_consensus.cc:2376] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8. Re-sending same reply.
I20260616 19:59:24.558898 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.542253 10969 raft_consensus.cc:740] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:24.562423 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:24.564106 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.564762 8044 raft_consensus.cc:1813] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Handling vote request from an unknown peer 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:24.565227 8044 raft_consensus.cc:2393] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:24.567651 8042 raft_consensus.cc:3060] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 12
I20260616 19:59:24.521368 11173 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.565022 8041 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:24.555814 11055 raft_consensus.cc:1064] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:59:24.570676 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:24.572391 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.572710 8113 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:24.585233 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.587800 8037 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:24.587013 7934 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:24.588305 8085 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:24.590438 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:24.591153 7961 raft_consensus.cc:2393] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:24.583900 7935 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:24.594386 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:24.594336 8038 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:24.595023 8115 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 12
I20260616 19:59:24.571527 11232 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:24.546059 11144 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.600875 8813 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:24.602414 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:24.604049 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:24.604111 11148 consensus_queue.cc:1059] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260616 19:59:24.605122 11148 raft_consensus.cc:3055] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:24.605436 11148 raft_consensus.cc:740] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:24.606701 8042 raft_consensus.cc:2468] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 12.
I20260616 19:59:24.606374 11148 consensus_queue.cc:260] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.607667 11148 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:24.607687 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.609675 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.610425 8042 raft_consensus.cc:2368] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6.
I20260616 19:59:24.611223 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.611629 8816 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:24.614511 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:24.618592 8011 leader_election.cc:400] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:24.619098 8011 leader_election.cc:403] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260616 19:59:24.580929 11055 raft_consensus.cc:1068] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Unable to promote non-voter c06f6f0858fe4ce6bd5b55efeb5ba2d8: Illegal state: Leader has not yet committed an operation in its own term
I20260616 19:59:24.623660 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.559829 11206 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.621366 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:24.621860 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 51005.7us
I20260616 19:59:24.625303 7935 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:24.626204 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.628894 8043 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:24.631325 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 54519.3us
I20260616 19:59:24.648000 8042 raft_consensus.cc:1240] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9. Ops: []
W20260616 19:59:24.668094 8009 leader_election.cc:336] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 1.508s (SENT)
I20260616 19:59:24.612267 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:24.675839 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.577898 8110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "303be6b16595410aae7147c6026435ff"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:24.634909 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:24.669369 11055 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:24.627107 11138 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:24.623237 11257 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.577109 11015 consensus_queue.cc:260] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.635644 11240 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:24.669741 8082 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:24.679224 8044 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:24.676999 8110 raft_consensus.cc:606] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:24.686705 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 116794us
I20260616 19:59:24.668658 8009 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:24.630297 11161 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:24.685238 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.673197 11263 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:24.616369 11232 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.637941 11162 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:24.681650 11068 consensus_queue.cc:1059] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260616 19:59:24.689839 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:24.691764 11138 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:24.712036 8042 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:24.713630 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:24.722052 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:24.722812 11147 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.723536 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.724622 8039 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:24.736892 7934 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:24.706038 8036 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
W20260616 19:59:24.719136 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 88467.2us
I20260616 19:59:24.746696 8038 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:24.717536 11015 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:24.709067 11162 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
W20260616 19:59:24.751878 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 72979.3us
I20260616 19:59:24.697791 11161 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.727643 11138 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:24.718657 11152 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:24.726048 11157 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
W20260616 19:59:24.762512 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 126631us
I20260616 19:59:24.739502 10998 raft_consensus.cc:2764] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260616 19:59:24.779850 8044 raft_consensus.cc:2468] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
W20260616 19:59:24.781814 11285 raft_consensus.cc:670] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:24.773918 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.756310 11162 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:24.788847 8082 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:24.751633 11263 raft_consensus.cc:697] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:59:24.794225 8009 leader_election.cc:336] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 1.555s (SENT)
I20260616 19:59:24.792788 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.789007 7933 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:24.777164 8813 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:24.708248 10969 consensus_queue.cc:260] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.801788 8042 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:24.803416 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:24.809862 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.810755 8042 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:24.813122 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 236615us
I20260616 19:59:24.814209 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.817276 8039 raft_consensus.cc:2393] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:24.817456 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:24.818765 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.784726 11157 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.819726 8042 raft_consensus.cc:2393] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 12: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:24.807386 11173 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:24.834049 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.832091 8042 raft_consensus.cc:1240] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:24.807626 11078 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:59:24.812283 11152 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:24.846297 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:24.846769 11292 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:24.804503 11240 consensus_queue.cc:1048] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260616 19:59:24.847409 11292 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.849725 8084 leader_election.cc:400] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12.
I20260616 19:59:24.850144 8084 leader_election.cc:403] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:59:24.850847 8084 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:24.854167 11258 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.854390 8038 raft_consensus.cc:1275] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260616 19:59:24.854830 11258 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:24.856448 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:24.857923 11152 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:24.858808 8044 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260616 19:59:24.860021 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:24.860770 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:24.806519 11246 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:24.849773 11292 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.869679 8084 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:24.872072 11277 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:24.873406 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:24.873024 11145 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.875121 11161 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:24.826710 11147 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:24.876013 11145 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.876091 11147 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.877249 11145 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:24.878530 11147 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:24.878623 11232 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:24.883169 8084 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:24.856523 8085 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:24.886555 8036 raft_consensus.cc:1240] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:24.888262 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:24.889011 8040 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:24.890976 8116 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:59:24.892621 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 102644us
I20260616 19:59:24.896046 11147 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:24.897531 8084 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:24.898463 11257 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:24.906454 8045 raft_consensus.cc:1275] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:59:24.906832 8084 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:24.907357 11284 log.cc:927] Time spent T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.280s user 0.000s sys 0.001s
I20260616 19:59:24.907855 8045 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:24.909884 11078 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:24.909945 7866 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: config changed from index 5 to 7, VOTER 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) evicted. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:24.911048 8084 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:24.913611 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55740 (request call id 228) took 9746 ms. Trace:
I20260616 19:59:24.823875 11144 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:24.920722 8042 raft_consensus.cc:1240] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260616 19:59:24.919754 8043 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:24.921729 8039 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:24.922423 8115 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 12.
I20260616 19:59:24.930498 8036 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:24.932309 8037 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:24.933074 8044 raft_consensus.cc:1240] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:24.866555 11258 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:24.933928 8114 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
W20260616 19:59:24.939883 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 105627us
I20260616 19:59:24.916095 10969 raft_consensus.cc:3060] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:59:24.966398 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 55458.2us
I20260616 19:59:24.967751 10998 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:24.885732 11070 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:24.969590 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 216721us
I20260616 19:59:24.906472 11257 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:24.923081 11078 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:24.933763 11158 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:24.984203 11263 consensus_queue.cc:237] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:24.988236 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 2048) took 1132 ms. Trace:
I20260616 19:59:24.988739 8043 rpcz_store.cc:276] 0616 19:59:23.848396 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.848892 (+ 496us) service_pool.cc:224] Handling call
0616 19:59:24.980422 (+1131530us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b54003acfd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:24.980589 (+ 167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1791416704}
I20260616 19:59:24.990075 8011 leader_election.cc:400] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:24.991513 8011 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:24.996317 11129 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:24.997088 11129 raft_consensus.cc:697] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:24.998324 11129 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:24.998960 11162 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.008217 11164 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:25.008797 11164 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.009490 11210 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:25.011194 11164 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:25.011495 11157 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:25.011770 11144 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:25.013206 7866 catalog_manager.cc:5696] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 7 to 10, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 10 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:59:24.968417 10998 raft_consensus.cc:697] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.021242 10998 consensus_queue.cc:237] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.024645 8084 leader_election.cc:400] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6.
W20260616 19:59:25.025120 11299 log.cc:927] Time spent T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260616 19:59:25.026638 11158 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:25.027158 11158 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:25.027403 8011 leader_election.cc:400] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 11. Current term is 12.
I20260616 19:59:25.027789 8011 leader_election.cc:403] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:59:24.964442 11293 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12.
I20260616 19:59:25.016036 7866 catalog_manager.cc:5696] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 4 to 6, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132). New cstate: current_term: 6 leader_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260616 19:59:25.030474 11078 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:25.030807 7866 catalog_manager.cc:5696] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:24.983949 11147 raft_consensus.cc:2804] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader election won for term 12
I20260616 19:59:25.040980 11147 raft_consensus.cc:697] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [term 12 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.041739 11162 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:25.025131 8084 leader_election.cc:403] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260616 19:59:25.048619 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 66560.8us
W20260616 19:59:25.053663 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:25.055563 8111 consensus_meta.cc:287] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.503s user 0.002s sys 0.006s
W20260616 19:59:25.057287 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.063709 8040 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
W20260616 19:59:25.049453 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 412948us
W20260616 19:59:25.067276 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 360848us
I20260616 19:59:25.056106 8111 raft_consensus.cc:2468] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
W20260616 19:59:25.066375 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 441067us
W20260616 19:59:25.073630 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 92653.7us
I20260616 19:59:25.074827 8084 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.073544 8085 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.065702 11144 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 11. Current term is 12.
I20260616 19:59:25.110649 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:25.110899 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:25.101052 11210 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:25.117136 8011 leader_election.cc:400] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
W20260616 19:59:25.113610 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.117517 8011 leader_election.cc:403] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260616 19:59:25.114059 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:25.114985 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.116297 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.042306 11147 consensus_queue.cc:237] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.101542 11145 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051) with higher term. Message: Invalid argument: T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6.
W20260616 19:59:25.133741 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 66464.5us
I20260616 19:59:25.135716 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1850) took 1248 ms. Trace:
I20260616 19:59:25.136618 8039 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:25.138046 8038 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:25.115948 11257 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:25.069900 11162 consensus_queue.cc:1059] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260616 19:59:25.143666 8085 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.136128 7968 rpcz_store.cc:276] 0616 19:59:23.879068 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:24.072490 (+193422us) service_pool.cc:224] Handling call
0616 19:59:24.097964 (+ 25474us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:24.169545 (+ 71581us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:24.169576 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:24.178865 (+ 9289us) log.cc:844] Serialized 26 byte log entry
0616 19:59:24.513231 (+334366us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:24.783360 (+270129us) log.cc:844] Serialized 14 byte log entry
0616 19:59:25.126757 (+343397us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:25.126857 (+ 100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:25.126931 (+ 74us) raft_consensus.cc:1685] finished
0616 19:59:25.126957 (+ 26us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:25.127243 (+ 286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17735,"prepare.queue_time_us":323431,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":184230,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":39985,"thread_start_us":943107,"threads_started":3,"wal-append.queue_time_us":439350}
I20260616 19:59:24.924048 7865 rpcz_store.cc:276] 0616 19:59:15.166920 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:15.167348 (+ 428us) service_pool.cc:224] Handling call
0616 19:59:15.321450 (+154102us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24f470 00007fe86f2ee31a 00007fe8750a3aac 00007fe8751a20ae 00007fe8736674a4 00007fe873667432 00007fe873667361 00007fe8736672dc 00007fe873666592 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:24.913365 (+9591915us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:23.438923 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:23.440995 (+ 2072us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:23.441049 (+ 54us) write_op.cc:435] Acquired schema lock
0616 19:59:23.441084 (+ 35us) tablet.cc:662] Decoding operations
0616 19:59:23.441648 (+ 564us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:23.441750 (+ 102us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:23.441793 (+ 43us) tablet.cc:685] Acquiring locks for 1 operations
0616 19:59:23.442111 (+ 318us) tablet.cc:701] Row locks acquired
0616 19:59:23.442145 (+ 34us) write_op.cc:260] PREPARE: finished
0616 19:59:23.442337 (+ 192us) write_op.cc:270] Start()
0616 19:59:23.442498 (+ 161us) write_op.cc:276] Timestamp: P: 1781639963442277 usec, L: 0
0616 19:59:23.442533 (+ 35us) op_driver.cc:348] REPLICATION: starting
0616 19:59:23.584906 (+142373us) log.cc:844] Serialized 536 byte log entry
0616 19:59:24.099508 (+514602us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e1208 00007fe872d1d401 00007fe872d1cbd8 00007fe872d21871 00007fe872d21829 00007fe872d217b9 00007fe872d21781 00007fe872d209ad 0000000000389354 0000000000389199
0616 19:59:24.345456 (+245948us) op_driver.cc:464] REPLICATION: finished
0616 19:59:24.881318 (+535862us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b54000acdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e270a 00007fe86f2682f4 00007fe86f2682a3 00007fe86f268218 00007fe86f2681d3 00007fe86f2674bc 00007fe86f2911f6 00007fe86f27909b 00007fe86f26377d 00007fe86f267155
0616 19:59:24.896196 (+ 14878us) write_op.cc:301] APPLY: starting
0616 19:59:24.896339 (+ 143us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:24.896490 (+ 151us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:24.896527 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:24.896843 (+ 316us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:24.897093 (+ 250us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0616 19:59:24.897143 (+ 50us) write_op.cc:312] APPLY: finished
0616 19:59:24.897435 (+ 292us) log.cc:844] Serialized 23 byte log entry
0616 19:59:24.897655 (+ 220us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:24.897746 (+ 91us) write_op.cc:454] Released schema lock
0616 19:59:24.897798 (+ 52us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:24.898255 (+ 457us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:24.898288 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9,"reported_tablets":11,"spinlock_wait_cycles":291006848,"child_traces":[["op",{"apply.queue_time_us":449981,"apply.run_cpu_time_us":2072,"apply.run_wall_time_us":65220,"num_ops":1,"prepare.queue_time_us":484927,"prepare.run_cpu_time_us":67090,"prepare.run_wall_time_us":739169,"raft.queue_time_us":419362,"raft.run_cpu_time_us":14314,"raft.run_wall_time_us":614017,"replication_time_us":882532,"spinlock_wait_cycles":89039616,"thread_start_us":1103836,"threads_started":4,"wal-append.queue_time_us":218583}]]}
I20260616 19:59:25.145215 7934 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.156924 11142 raft_consensus.cc:3055] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:25.157467 11142 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:25.160511 8113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:25.161257 8113 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.119704 11157 consensus_queue.cc:1059] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260616 19:59:25.139827 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:25.119140 11138 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:25.146224 11148 consensus_meta.cc:287] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.538s user 0.011s sys 0.001s
W20260616 19:59:25.168465 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 53584.4us
W20260616 19:59:25.169075 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.142253 11210 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:25.146631 11144 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:25.163213 11145 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
W20260616 19:59:25.174664 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 58433.9us
W20260616 19:59:25.188774 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 48750.3us
I20260616 19:59:25.158314 11142 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.199154 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.202210 7935 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.176980 11295 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:25.208900 11142 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:25.209048 11300 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:25.213203 8084 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.213867 8811 raft_consensus.cc:2410] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260616 19:59:25.214386 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.216084 11206 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:25.187351 11218 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260616 19:59:25.221274 11218 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260616 19:59:25.221681 11218 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:25.222606 11218 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.223762 11218 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:25.224267 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:25.229919 8011 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:25.230669 11144 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:25.231257 11277 raft_consensus.cc:3055] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:25.231666 11277 raft_consensus.cc:740] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.232558 11277 consensus_queue.cc:260] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.233008 11070 consensus_queue.cc:1059] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:25.234017 11068 raft_consensus.cc:3055] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:25.234570 11068 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:25.234664 11294 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:25.235186 11294 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.235492 11068 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.240999 11068 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:25.242321 11055 raft_consensus.cc:2764] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260616 19:59:25.234051 11277 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:25.204881 11297 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:25.237630 8084 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.244367 11297 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:25.244776 11297 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:25.245539 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.231918 11144 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:25.250202 11263 raft_consensus.cc:2804] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:25.250698 11263 raft_consensus.cc:697] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.251560 11263 consensus_queue.cc:237] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.236953 11301 raft_consensus.cc:1064] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
I20260616 19:59:25.254833 11257 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:25.239427 7935 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.191641 11080 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260616 19:59:25.277237 11080 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:25.277666 11080 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:25.278537 11080 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.279816 11080 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:25.240326 8085 leader_election.cc:400] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Vote denied by peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531) with higher term. Message: Invalid argument: T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12.
W20260616 19:59:25.280206 11309 log.cc:927] Time spent T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.082s user 0.000s sys 0.001s
I20260616 19:59:25.211050 11300 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:25.281369 11300 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:25.264518 11301 raft_consensus.cc:1068] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Unable to promote non-voter c06f6f0858fe4ce6bd5b55efeb5ba2d8: Illegal state: Leader has not yet committed an operation in its own term
I20260616 19:59:25.282632 11301 raft_consensus.cc:2804] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:25.283221 11301 raft_consensus.cc:697] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:25.284011 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.210319 11145 raft_consensus.cc:697] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.284495 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.284101 11301 consensus_queue.cc:237] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.238118 11157 raft_consensus.cc:2804] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:25.285485 11157 raft_consensus.cc:697] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.285244 11145 consensus_queue.cc:237] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.285844 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.286494 7961 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:25.270176 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:25.286381 11157 consensus_queue.cc:237] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.287283 8038 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:25.269699 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:25.210104 11258 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:25.280259 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.259621 11210 consensus_queue.cc:1059] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260616 19:59:25.277158 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:25.292433 11015 consensus_meta.cc:287] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.533s user 0.003s sys 0.006s
I20260616 19:59:25.293038 11015 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election won for term 12
I20260616 19:59:25.293509 11015 raft_consensus.cc:697] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:25.293581 8113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9837c086539a4592bb5aada5872652ca"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:25.294224 11295 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260616 19:59:25.294423 11015 consensus_queue.cc:237] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.284811 7968 raft_consensus.cc:2393] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 12: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:25.294224 8113 raft_consensus.cc:606] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:25.295933 8113 raft_consensus.cc:614] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:25.297053 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.292501 11258 raft_consensus.cc:3055] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:25.297533 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.282694 11300 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:25.299532 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.297669 11258 raft_consensus.cc:740] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.300334 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.300938 7962 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:25.300678 11258 consensus_queue.cc:260] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.302011 11258 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:25.304530 8113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:25.304916 7964 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:25.305078 8113 raft_consensus.cc:606] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:25.238118 11138 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:25.305516 8113 raft_consensus.cc:614] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:25.306178 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.306572 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.306994 7968 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 11: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:25.271768 11294 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:25.243793 11078 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.308112 6782 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260616 19:59:25.308753 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 52821.4us
I20260616 19:59:25.294814 7935 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.313925 7962 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:25.312595 7935 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.317437 8041 raft_consensus.cc:2393] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:25.318917 8038 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
W20260616 19:59:25.331064 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 189240us
I20260616 19:59:25.341740 7968 raft_consensus.cc:1240] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:25.346288 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.347927 7934 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.327414 11138 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:25.350023 8084 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.317145 11144 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260616 19:59:25.317173 11210 consensus_queue.cc:1059] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260616 19:59:25.337826 11055 consensus_queue.cc:1059] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260616 19:59:25.318041 10998 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260616 19:59:25.345816 8009 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:25.324343 10969 consensus_queue.cc:1048] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260616 19:59:25.318297 11155 consensus_queue.cc:1048] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260616 19:59:25.340862 11161 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:25.335404 11240 consensus_queue.cc:1048] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260616 19:59:25.335117 11232 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260616 19:59:25.336465 11158 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.366739 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:25.367024 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:25.367923 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:25.368176 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:25.368448 8036 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:59:25.371500 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:25.371614 8038 raft_consensus.cc:1240] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:25.372566 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:25.373309 8044 raft_consensus.cc:2393] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:25.374351 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:25.374970 8038 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:59:25.345572 11021 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260616 19:59:25.385031 11312 consensus_queue.cc:1059] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260616 19:59:25.388725 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.389133 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.389272 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.389437 7965 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:25.392606 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.392828 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:25.393610 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.395303 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:25.396354 8044 raft_consensus.cc:1240] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260616 19:59:25.363603 11263 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:25.392124 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.401360 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.402647 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.403319 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.411707 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.365136 11301 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:25.412500 8115 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:25.412892 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:25.419106 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:25.419241 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 73463.9us
W20260616 19:59:25.421244 8009 leader_election.cc:336] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 1.590s (SENT)
I20260616 19:59:25.421663 8009 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:25.423707 7961 raft_consensus.cc:1240] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:25.386166 11078 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:25.432499 7970 raft_consensus.cc:1240] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260616 19:59:25.426190 7964 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:25.427109 7968 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:25.429502 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.435341 8119 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:25.388710 11293 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:25.422453 8785 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.436116 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.382294 11162 consensus_queue.cc:1059] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260616 19:59:25.443310 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.439909 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.380142 11297 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.375778 11161 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.447314 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.446601 7967 raft_consensus.cc:1275] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260616 19:59:25.447654 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.385363 11246 consensus_queue.cc:1059] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260616 19:59:25.449819 7963 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:25.452785 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.456821 8113 raft_consensus.cc:2393] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:25.448397 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.511637 8115 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 10: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:25.457263 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.464622 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.513926 8110 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:25.515622 8110 raft_consensus.cc:1240] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260616 19:59:25.516711 8110 raft_consensus.cc:1240] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:25.521759 8110 raft_consensus.cc:1240] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: []
W20260616 19:59:25.522966 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:25.550951 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.531785 7964 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:25.548939 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.553421 8115 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:25.436168 11302 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260616 19:59:25.443207 11293 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.445680 11206 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:25.523598 11312 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:25.531322 11155 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260616 19:59:25.566494 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.575551 8110 raft_consensus.cc:2410] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260616 19:59:25.578166 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.579656 8115 raft_consensus.cc:2410] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260616 19:59:25.580500 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:25.580972 11314 log.cc:927] Time spent T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260616 19:59:25.581951 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.585378 7969 raft_consensus.cc:1240] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:25.587260 8116 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:25.587745 7961 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:59:25.588130 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:25.588536 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:25.594906 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 170810us
I20260616 19:59:25.559805 11144 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:25.563740 11317 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:25.599634 8118 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:25.612291 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.613914 11055 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:25.614432 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 263855us
I20260616 19:59:25.457918 11078 consensus_queue.cc:1059] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260616 19:59:25.625566 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:25.572302 11312 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:25.628022 8011 leader_election.cc:336] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.794s (SENT)
I20260616 19:59:25.627513 8784 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:25.594161 11263 raft_consensus.cc:697] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:59:25.681301 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.659845 7935 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:25.630925 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 283027us
W20260616 19:59:25.642967 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:25.753716 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.862479 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.874276 8110 raft_consensus.cc:2368] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
W20260616 19:59:25.736279 8111 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 2231) took 1792 ms (client timeout 2241 ms). Trace:
I20260616 19:59:25.628888 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:25.661548 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.875909 8118 raft_consensus.cc:2410] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260616 19:59:25.661037 8115 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:25.880071 11164 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260616 19:59:25.880388 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 252941us
I20260616 19:59:25.687248 7965 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 11.
I20260616 19:59:25.880595 11164 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:25.728363 6782 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260616 19:59:25.599124 11206 raft_consensus.cc:697] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:25.739006 11295 raft_consensus.cc:3055] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:25.909364 11295 raft_consensus.cc:740] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:25.896763 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:25.875185 8111 rpcz_store.cc:269] 0616 19:59:23.943770 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:23.958938 (+ 15168us) service_pool.cc:224] Handling call
0616 19:59:25.736017 (+1777079us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b5400102f54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:25.736192 (+ 175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177563904}
I20260616 19:59:25.598585 11240 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:25.615509 11138 raft_consensus.cc:697] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:25.794023 11257 consensus_meta.cc:287] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.517s user 0.008s sys 0.001s
W20260616 19:59:25.881712 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 255586us
I20260616 19:59:25.892177 8784 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:25.925334 7935 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:25.741552 11293 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:25.793828 11277 consensus_meta.cc:287] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.551s user 0.009s sys 0.001s
W20260616 19:59:25.740794 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:25.876477 11300 consensus_meta.cc:287] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.578s user 0.010s sys 0.000s
I20260616 19:59:25.738454 10998 raft_consensus.cc:3055] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:25.841187 11297 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:25.914402 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 286379us
W20260616 19:59:25.936189 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44114.6us
I20260616 19:59:25.683406 11161 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:25.688900 11313 log.cc:927] Time spent T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.071s user 0.000s sys 0.000s
I20260616 19:59:25.905737 11325 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election won for term 8
I20260616 19:59:25.737733 11021 raft_consensus.cc:3055] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:25.926868 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.943665 8111 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:25.782755 11158 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:25.620298 11317 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:25.971259 11158 raft_consensus.cc:2804] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:25.934976 11312 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.971868 11317 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.973273 11317 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:25.973344 11312 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:25.971900 11158 raft_consensus.cc:697] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:25.977559 11158 consensus_queue.cc:237] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:25.978572 11315 log.cc:927] Time spent T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.579s user 0.004s sys 0.000s
I20260616 19:59:25.931095 7968 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:25.929116 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.932464 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.982537 8110 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:25.896930 8811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52470 (request call id 155) took 1274 ms. Trace:
I20260616 19:59:25.934016 11295 consensus_queue.cc:260] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:25.934444 11301 consensus_queue.cc:1059] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260616 19:59:25.927311 11257 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:25.934607 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.914638 11164 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:25.939301 8114 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:26.022122 8116 raft_consensus.cc:2368] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:26.023185 8113 raft_consensus.cc:1240] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:26.023267 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:25.946348 11138 consensus_queue.cc:237] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.024964 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:26.026371 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.937938 10998 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:25.948267 11325 raft_consensus.cc:697] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:26.028515 10998 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.028682 11325 consensus_queue.cc:237] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.029663 10998 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:25.908869 11263 consensus_queue.cc:237] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:25.908823 11319 consensus_queue.cc:1059] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260616 19:59:26.034474 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:26.032939 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 124607us
I20260616 19:59:26.035199 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:25.906822 11162 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260616 19:59:25.943326 11021 raft_consensus.cc:740] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:25.957445 11206 consensus_queue.cc:237] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:25.961886 11258 consensus_meta.cc:287] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.659s user 0.005s sys 0.005s
I20260616 19:59:25.722174 11322 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.247s
I20260616 19:59:25.970526 11285 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:25.986485 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:25.945039 11218 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:25.989050 8040 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260616 19:59:26.015249 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.667938 11055 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.044802 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:26.032423 8037 raft_consensus.cc:1275] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260616 19:59:25.981199 11316 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.432s user 0.002s sys 0.000s
I20260616 19:59:26.033252 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:26.040020 8110 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 10: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:25.988526 8041 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260616 19:59:26.042348 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 117121us
I20260616 19:59:26.027655 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:26.048986 7935 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:26.049971 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:25.938767 11240 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.053031 8119 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
I20260616 19:59:26.056106 7934 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:26.059808 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:26.059832 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:26.064067 8038 raft_consensus.cc:2393] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:25.988498 8811 rpcz_store.cc:276] 0616 19:59:24.622311 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:24.623591 (+ 1280us) service_pool.cc:224] Handling call
0616 19:59:25.896681 (+1273090us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54005b13d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:25.896843 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098772352}
W20260616 19:59:26.086103 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.093859 11344 raft_consensus.cc:3055] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:26.093712 11021 consensus_queue.cc:260] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.094619 11344 raft_consensus.cc:740] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:26.096141 11330 consensus_queue.cc:1059] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260616 19:59:26.095741 11344 consensus_queue.cc:260] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.113255 11301 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:26.113991 11301 raft_consensus.cc:697] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:26.115142 11301 consensus_queue.cc:237] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.118479 7963 raft_consensus.cc:1240] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:26.118700 11337 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260616 19:59:26.128248 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 79325.6us
W20260616 19:59:26.130955 11129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s user 0.006s sys 0.009s
I20260616 19:59:26.139675 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:26.054816 11285 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:26.141935 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 76977.4us
I20260616 19:59:26.163311 8038 raft_consensus.cc:1240] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260616 19:59:26.143113 8110 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:26.152355 8040 raft_consensus.cc:1275] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:59:26.152801 8042 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:26.156366 8117 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:59:26.156949 11232 consensus_queue.cc:1059] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:59:26.165956 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:26.166317 11232 raft_consensus.cc:3055] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:26.166759 11232 raft_consensus.cc:740] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:26.166970 11240 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:26.167745 11232 consensus_queue.cc:260] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.169059 11232 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
W20260616 19:59:26.159523 11340 log.cc:927] Time spent T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.103s user 0.000s sys 0.000s
I20260616 19:59:26.164227 11277 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260616 19:59:26.173079 11055 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:26.149528 11325 consensus_queue.cc:1175] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6827184e3cef42c2ada15b2d31afee4b" responder_term: 4 status { last_received { term: 3 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260616 19:59:26.181074 8038 raft_consensus.cc:1240] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:26.175544 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:26.179471 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:26.181895 8042 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:26.185537 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:26.147462 11129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s user 0.007s sys 0.009s
I20260616 19:59:26.193531 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:26.149930 11344 raft_consensus.cc:3060] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:26.175442 11295 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:26.209496 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:26.209530 7962 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:26.210635 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:26.211143 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:26.211733 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:26.207603 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:26.212575 8042 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:26.213748 8042 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:26.215540 7966 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260616 19:59:26.215628 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:26.209604 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:26.217787 7961 raft_consensus.cc:2393] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:26.224326 7967 raft_consensus.cc:1240] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:59:26.223831 8042 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:59:26.234951 8042 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260616 19:59:26.236443 11346 consensus_queue.cc:1059] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260616 19:59:26.240585 8042 raft_consensus.cc:1240] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:26.241164 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:26.243089 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:26.243162 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:26.229193 11341 log.cc:927] Time spent T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.075s user 0.001s sys 0.001s
I20260616 19:59:26.225068 11312 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:26.207983 11162 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260616 19:59:26.258121 8011 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:26.221228 11277 consensus_queue.cc:1048] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260616 19:59:26.255673 11337 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:26.281608 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:26.280084 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.258802 11312 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:26.282052 11337 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:26.289222 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.256345 11138 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260616 19:59:26.265983 11320 consensus_queue.cc:1059] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260616 19:59:26.311380 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:26.259128 11319 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260616 19:59:26.286155 11312 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:26.316301 11328 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:59:26.316504 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.297210 11337 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:26.317684 7962 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260616 19:59:26.319259 7962 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260616 19:59:26.321548 8784 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:26.323298 11080 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:26.323683 11080 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:26.324034 11080 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:26.275281 11293 consensus_queue.cc:1059] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260616 19:59:26.332837 8044 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260616 19:59:26.334125 8044 pending_rounds.cc:85] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 5
I20260616 19:59:26.334275 7967 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:26.286527 11240 consensus_queue.cc:1059] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260616 19:59:26.334398 8044 pending_rounds.cc:107] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260616 19:59:26.337698 8044 raft_consensus.cc:2889] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260616 19:59:26.335269 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 61168us
W20260616 19:59:26.344388 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 244886us
I20260616 19:59:26.346464 8011 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:26.368754 7934 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:26.363279 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:26.348613 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.372603 8011 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:26.366252 11129 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:26.382753 8011 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:26.374190 11232 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:26.346827 11317 consensus_queue.cc:1059] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260616 19:59:26.390416 11357 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260616 19:59:26.402529 11162 raft_consensus.cc:1064] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: attempting to promote NON_VOTER c06f6f0858fe4ce6bd5b55efeb5ba2d8 to VOTER
W20260616 19:59:26.403208 11339 log.cc:927] Time spent T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.124s user 0.000s sys 0.001s
I20260616 19:59:26.344467 11164 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:26.411448 8119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1953) took 1003 ms. Trace:
I20260616 19:59:26.412851 11312 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.394259 11142 raft_consensus.cc:3055] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:26.389381 11129 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:26.425185 11306 log.cc:927] Time spent T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 1.199s user 0.004s sys 0.001s
I20260616 19:59:26.425603 11162 consensus_queue.cc:237] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 8, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:26.428828 8112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 265) took 1852 ms. Trace:
I20260616 19:59:26.429742 11164 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:26.362867 11319 consensus_queue.cc:1048] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260616 19:59:26.419633 8119 rpcz_store.cc:276] 0616 19:59:25.407611 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:25.443060 (+ 35449us) service_pool.cc:224] Handling call
0616 19:59:26.411255 (+968195us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54002ccc54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:26.411384 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400673664}
I20260616 19:59:26.417624 11161 raft_consensus.cc:2749] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:26.421774 11246 raft_consensus.cc:3055] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:26.431927 11246 raft_consensus.cc:740] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:26.389338 11356 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:26.419063 11070 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:26.422705 11142 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:26.429320 8112 rpcz_store.cc:276] 0616 19:59:24.576279 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:24.576416 (+ 137us) service_pool.cc:224] Handling call
0616 19:59:24.587224 (+ 10808us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:24.801831 (+214607us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:24.801861 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:24.809966 (+ 8105us) log.cc:844] Serialized 26 byte log entry
0616 19:59:25.158626 (+348660us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:25.158662 (+ 36us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:25.158740 (+ 78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:26.424545 (+1265805us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b440002250c. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c140375 00007fe86c140d92 00007fe86ee9057b 00007fe86ee8be4f 00007fe86ee7fc0e 00007fe86ee7ef56 00007fe86ee7eb78 00007fe86ee885c1 00007fe86ee88579 00007fe86ee88509
0616 19:59:26.428448 (+ 3903us) raft_consensus.cc:1685] finished
0616 19:59:26.428472 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:26.428738 (+ 266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1948,"raft.queue_time_us":22759,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7523,"spinlock_wait_cycles":28677504,"thread_start_us":469757,"threads_started":2,"wal-append.queue_time_us":380462}
W20260616 19:59:26.421031 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 73171.2us
W20260616 19:59:26.434468 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.455677 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:26.432884 11246 consensus_queue.cc:260] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:26.465097 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 82423.3us
I20260616 19:59:26.472936 7934 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:26.479737 11142 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.423270 11337 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.484077 7934 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:26.475584 11335 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:26.463888 11070 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:26.515506 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.515897 11336 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:26.520327 11366 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260616 19:59:26.524261 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.523965 7934 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:26.530381 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 58137.1us
I20260616 19:59:26.550959 7934 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:26.557337 8011 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:26.552460 11319 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:26.557360 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:26.561677 11329 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.050s user 0.000s sys 0.001s
I20260616 19:59:26.561793 11365 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election won for term 9
W20260616 19:59:26.562376 11362 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.106s user 0.000s sys 0.001s
I20260616 19:59:26.566057 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:26.566356 11263 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:26.566648 8042 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:26.566915 11263 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:26.541817 11080 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:26.571585 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.575410 8042 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:26.559600 11297 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:26.581390 8042 raft_consensus.cc:1275] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260616 19:59:26.581790 8042 pending_rounds.cc:85] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 2
I20260616 19:59:26.582000 8042 pending_rounds.cc:107] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260616 19:59:26.582217 8042 raft_consensus.cc:2889] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:26.583652 8042 raft_consensus.cc:1275] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260616 19:59:26.560845 11319 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:26.588526 7934 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:26.574846 11015 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:26.562417 11365 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:26.581461 11297 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
W20260616 19:59:26.608001 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.599792 11319 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:59:26.798784 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:26.799369 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:26.799917 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 211459us
I20260616 19:59:26.601593 11312 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:26.802428 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:26.805895 7934 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:26.806607 8037 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:26.819648 11378 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:26.820291 11378 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:26.825291 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:26.629400 11297 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:26.792380 11326 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:26.835641 7965 raft_consensus.cc:1275] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260616 19:59:26.835770 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:26.838263 7967 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:26.847023 7963 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:59:26.832746 11337 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:26.836849 11070 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:26.871827 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.877424 8037 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:26.882936 11191 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:26.883654 11191 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:26.858924 11326 raft_consensus.cc:697] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:26.851480 11312 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:26.937744 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:26.938467 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:26.945175 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.894995 11070 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:26.949427 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:26.949678 11070 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:26.950165 8811 raft_consensus.cc:1813] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Handling vote request from an unknown peer 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:26.950899 8811 raft_consensus.cc:2410] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260616 19:59:26.951411 11161 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:26.901741 11371 log.cc:927] Time spent T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.292s user 0.001s sys 0.000s
I20260616 19:59:26.877333 11263 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:26.947131 7935 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:26.877117 11311 log.cc:927] Time spent T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 1.186s user 0.001s sys 0.004s
I20260616 19:59:26.954993 7935 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:26.910125 11232 consensus_meta.cc:287] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.505s user 0.008s sys 0.000s
I20260616 19:59:26.955453 7962 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260616 19:59:26.900425 11369 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.102s user 0.001s sys 0.000s
W20260616 19:59:26.899583 11350 log.cc:927] Time spent T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.187s user 0.000s sys 0.000s
W20260616 19:59:26.901057 11372 log.cc:927] Time spent T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
W20260616 19:59:26.910722 11295 consensus_meta.cc:287] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.692s user 0.011s sys 0.000s
W20260616 19:59:26.957304 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:26.885015 11285 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:26.908450 11080 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:26.909837 11334 raft_consensus.cc:3055] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:26.905202 11302 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:26.952533 11070 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:26.978302 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 172497us
I20260616 19:59:26.907990 11129 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:26.947687 11378 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:26.951924 11161 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:26.974167 11337 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:26.997861 7934 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:26.975545 11232 raft_consensus.cc:2749] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:27.000501 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 210359us
I20260616 19:59:26.877676 11365 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.044710 7964 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:27.045686 11129 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:27.046367 11129 raft_consensus.cc:697] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:26.999884 11302 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.017067 11384 raft_consensus.cc:2804] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:26.976737 11325 consensus_queue.cc:1048] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260616 19:59:26.990232 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:27.002027 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:26.996547 11334 raft_consensus.cc:740] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:26.313367 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260616 19:59:27.133193 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 178245us
I20260616 19:59:27.162635 8044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2536) took 1176 ms. Trace:
W20260616 19:59:27.106623 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.164239 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 166460us
I20260616 19:59:27.190404 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:27.191071 7967 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:27.192015 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:27.192797 7967 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:27.187587 8044 rpcz_store.cc:276] 0616 19:59:25.986270 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:25.987197 (+ 927us) service_pool.cc:224] Handling call
0616 19:59:25.987599 (+ 402us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:26.340887 (+353288us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:27.162274 (+821387us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54003acad4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:27.162549 (+ 275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793073536}
I20260616 19:59:27.201515 7967 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: []
W20260616 19:59:27.081341 11350 log.cc:927] Time spent T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
W20260616 19:59:27.089138 11335 consensus_meta.cc:287] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.587s user 0.001s sys 0.009s
I20260616 19:59:27.224651 7963 raft_consensus.cc:1240] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260616 19:59:27.206236 8818 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:27.233043 11381 consensus_queue.cc:1059] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260616 19:59:26.973237 11326 consensus_queue.cc:237] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.085022 11337 raft_consensus.cc:697] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:27.234921 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:27.235118 8008 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:27.235242 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:27.235844 8085 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:27.242537 8117 raft_consensus.cc:1275] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260616 19:59:27.248668 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.221616 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.252065 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.253942 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.254962 8119 raft_consensus.cc:2393] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 11: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:27.206454 11080 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:27.147058 11384 raft_consensus.cc:697] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:27.258545 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.266985 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:27.259210 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 158383us
I20260616 19:59:27.260475 8111 raft_consensus.cc:1275] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260616 19:59:27.260682 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.270485 8110 raft_consensus.cc:2410] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260616 19:59:27.261778 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:27.267597 8116 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:27.271675 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.271505 11080 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:27.260896 11378 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:27.236307 11335 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:27.272665 11378 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:27.273092 11378 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:27.273872 11080 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:27.274258 11384 consensus_queue.cc:237] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.277981 11379 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260616 19:59:27.278187 11161 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:27.278422 11379 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:27.279235 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.284497 8117 raft_consensus.cc:2410] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260616 19:59:27.266320 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.285444 8119 raft_consensus.cc:2393] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:27.047464 11129 consensus_queue.cc:237] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.303179 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.257280 11285 raft_consensus.cc:3055] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Stepping down as leader of term 8
W20260616 19:59:27.307331 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.303880 8117 raft_consensus.cc:2376] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9. Re-sending same reply.
I20260616 19:59:27.288317 11263 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:27.366281 11263 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.302580 8011 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:27.288554 11379 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:27.373343 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:27.287824 11210 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:27.302177 11383 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:27.375325 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:27.622602 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.717351 8117 raft_consensus.cc:2393] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:27.353518 11080 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:27.638397 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.734635 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.447264 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.311477 11285 raft_consensus.cc:740] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
W20260616 19:59:27.453074 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.715987 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 480172us
W20260616 19:59:27.736840 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.537935 8786 leader_election.cc:336] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.658s (SENT)
I20260616 19:59:27.715471 8111 raft_consensus.cc:2468] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:27.300455 11161 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:27.730104 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.388604 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 86441.1us
I20260616 19:59:27.737471 8786 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:27.756947 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.758663 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.759446 8117 raft_consensus.cc:2468] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
W20260616 19:59:27.773228 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45978.1us
W20260616 19:59:27.776000 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.784217 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.785632 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:27.786252 8117 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:27.789812 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:27.790455 8111 raft_consensus.cc:2376] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8. Re-sending same reply.
I20260616 19:59:27.400949 11191 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:27.388991 11387 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:27.812173 11387 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:27.812740 11387 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:27.813496 11386 consensus_queue.cc:1059] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260616 19:59:27.280776 11334 consensus_queue.cc:260] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.733042 11210 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.817198 11334 raft_consensus.cc:3060] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:27.818807 11210 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:27.781932 8009 connection.cc:256] RPC call timeout handler was delayed by -2.1244996890000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260616 19:59:27.404112 11302 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:27.821442 8039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s user 0.022s sys 0.035s
W20260616 19:59:27.822448 8039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s user 0.022s sys 0.035s
W20260616 19:59:27.826049 8040 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s user 0.029s sys 0.116s
I20260616 19:59:27.740613 11365 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:27.826503 8040 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.570s user 0.029s sys 0.116s
W20260616 19:59:27.831845 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:27.834517 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 62649.5us
I20260616 19:59:27.838444 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:27.839525 8111 raft_consensus.cc:2410] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260616 19:59:27.820825 11356 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:27.829748 11021 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:27.820569 11191 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:27.772241 11080 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:27.782488 11378 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:27.916467 8111 raft_consensus.cc:1275] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260616 19:59:27.868757 11295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s user 0.056s sys 0.060s
W20260616 19:59:27.936733 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 194310us
W20260616 19:59:27.940408 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.775022 11161 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:27.917126 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 81721.5us
I20260616 19:59:27.616354 11337 consensus_queue.cc:237] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:27.946250 8118 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1942) took 2543 ms (client timeout 2107 ms). Trace:
W20260616 19:59:27.946736 8118 rpcz_store.cc:269] 0616 19:59:25.403162 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:25.436137 (+ 32975us) service_pool.cc:224] Handling call
0616 19:59:27.946014 (+2509877us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b5400438d54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:27.946172 (+ 158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204559616}
I20260616 19:59:27.949728 8085 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:27.944022 11295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s user 0.057s sys 0.061s
I20260616 19:59:27.964272 8111 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260616 19:59:27.842226 11365 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.733266 11383 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:27.973769 8111 pending_rounds.cc:85] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 4
I20260616 19:59:27.981178 8111 pending_rounds.cc:107] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260616 19:59:27.981560 8111 raft_consensus.cc:2889] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:27.857419 11285 consensus_queue.cc:260] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:27.988907 11285 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:27.964329 11381 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:59:27.996436 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 46824.1us
I20260616 19:59:27.973160 11323 raft_consensus.cc:3055] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:27.997742 8085 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:27.950855 11191 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:27.998893 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:27.874245 11356 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:28.077209 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 79579.3us
W20260616 19:59:28.034303 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 109462us
I20260616 19:59:28.012768 11323 raft_consensus.cc:740] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:28.109843 8085 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.113808 7967 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 10. Current term is 11. Ops: []
I20260616 19:59:28.079602 11263 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:28.081897 11210 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:28.116318 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.117240 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.118845 8117 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 11.
I20260616 19:59:28.119208 8085 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.087141 11317 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.088372 11393 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.121068 7963 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 10. Current term is 11. Ops: []
I20260616 19:59:28.120466 11302 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260616 19:59:28.129976 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.134635 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.137619 8085 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:28.139637 11398 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.142s user 0.000s sys 0.001s
W20260616 19:59:28.140563 11405 log.cc:927] Time spent T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260616 19:59:28.132764 11302 raft_consensus.cc:2749] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:28.144516 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.144927 11302 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.145704 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.131608 11393 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.146780 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.148125 11393 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.145449 11302 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.150146 11302 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:28.150660 11302 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:28.147876 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.151626 7963 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:28.152583 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.160459 7963 raft_consensus.cc:2376] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8. Re-sending same reply.
W20260616 19:59:28.160754 11406 log.cc:927] Time spent T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.123s user 0.000s sys 0.000s
W20260616 19:59:28.161856 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.161979 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2544) took 1744 ms. Trace:
I20260616 19:59:28.162618 7963 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260616 19:59:28.162534 8039 rpcz_store.cc:276] 0616 19:59:26.417308 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:26.527140 (+109832us) service_pool.cc:224] Handling call
0616 19:59:26.527605 (+ 465us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:26.989517 (+461912us) raft_consensus.cc:1534] Early marking committed up to index 1
0616 19:59:26.989794 (+ 277us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:27.001027 (+ 11233us) log.cc:844] Serialized 26 byte log entry
0616 19:59:27.822862 (+821835us) raft_consensus.cc:1645] Marking committed up to 1
0616 19:59:27.822929 (+ 67us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:27.823021 (+ 92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:28.161681 (+338660us) raft_consensus.cc:1685] finished
0616 19:59:28.161725 (+ 44us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:28.161913 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2706,"raft.queue_time_us":24926,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2291,"thread_start_us":1167018,"threads_started":2,"wal-append.queue_time_us":938262}
W20260616 19:59:28.164794 11407 log.cc:927] Time spent T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260616 19:59:28.166905 11191 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.123693 11263 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:28.179105 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:28.121681 11078 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:28.185266 11383 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.185521 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2564) took 1384 ms. Trace:
I20260616 19:59:28.186034 11383 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:28.186098 7962 rpcz_store.cc:276] 0616 19:59:26.801297 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:26.824621 (+ 23324us) service_pool.cc:224] Handling call
0616 19:59:26.844477 (+ 19856us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:26.959118 (+114641us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:28.185209 (+1226091us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540016ba54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:28.185438 (+ 229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254409472}
I20260616 19:59:28.186791 11383 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:28.192512 11399 log.cc:927] Time spent T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.267s user 0.000s sys 0.001s
I20260616 19:59:28.193401 11378 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:28.194002 11157 raft_consensus.cc:3055] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:28.194408 11157 raft_consensus.cc:740] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:28.195502 11080 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.195353 11157 consensus_queue.cc:260] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:28.196545 11394 log.cc:927] Time spent T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.273s user 0.000s sys 0.001s
I20260616 19:59:28.128178 11317 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.132058 11210 raft_consensus.cc:697] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:28.201794 11210 consensus_queue.cc:237] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.203243 7866 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59082 (request call id 101) took 7041 ms. Trace:
W20260616 19:59:28.131320 11319 consensus_meta.cc:287] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 1.325s user 0.013s sys 0.000s
I20260616 19:59:28.214542 7934 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:28.199982 11317 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:28.210902 8811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52470 (request call id 162) took 1254 ms. Trace:
W20260616 19:59:28.217638 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 679714us
I20260616 19:59:28.222350 11335 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.218125 8811 rpcz_store.cc:276] 0616 19:59:26.955842 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:26.958491 (+ 2649us) service_pool.cc:224] Handling call
0616 19:59:28.178877 (+1220386us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b5800003998. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6572 00007fe8750d97d9 00007fe8742345e0 00007fe8742fd7ab 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:28.209480 (+ 30603us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8472448}
I20260616 19:59:28.223058 11335 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.225261 11335 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:28.225813 11297 consensus_meta.cc:287] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 1.381s user 0.006s sys 0.005s
I20260616 19:59:28.226243 11297 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.214474 11319 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.228078 7965 raft_consensus.cc:1240] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:28.228935 11319 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.229645 7967 raft_consensus.cc:1240] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:28.230866 11021 raft_consensus.cc:2749] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:28.231606 11392 consensus_queue.cc:1048] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:59:28.232398 7934 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.233079 11392 consensus_queue.cc:1048] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:28.234567 11161 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.237165 11356 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.132732 11323 consensus_queue.cc:260] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.237749 11320 raft_consensus.cc:3055] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:28.238250 11320 raft_consensus.cc:740] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:28.238524 11323 raft_consensus.cc:3060] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:28.239187 11320 consensus_queue.cc:260] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.239703 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.240967 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.203720 7866 rpcz_store.cc:276] 0616 19:59:21.161776 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:21.167980 (+ 6204us) service_pool.cc:224] Handling call
0616 19:59:28.203139 (+7035159us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:25.240028 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:25.240593 (+ 565us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:25.240648 (+ 55us) write_op.cc:435] Acquired schema lock
0616 19:59:25.240667 (+ 19us) tablet.cc:662] Decoding operations
0616 19:59:25.241250 (+ 583us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:25.241323 (+ 73us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:25.241349 (+ 26us) tablet.cc:685] Acquiring locks for 4 operations
0616 19:59:25.241718 (+ 369us) tablet.cc:701] Row locks acquired
0616 19:59:25.241736 (+ 18us) write_op.cc:260] PREPARE: finished
0616 19:59:25.241863 (+ 127us) write_op.cc:270] Start()
0616 19:59:25.241976 (+ 113us) write_op.cc:276] Timestamp: P: 1781639965241821 usec, L: 0
0616 19:59:25.241999 (+ 23us) op_driver.cc:348] REPLICATION: starting
0616 19:59:25.242622 (+ 623us) log.cc:844] Serialized 1686 byte log entry
0616 19:59:25.244955 (+ 2333us) op_driver.cc:464] REPLICATION: finished
0616 19:59:25.247081 (+ 2126us) write_op.cc:301] APPLY: starting
0616 19:59:25.247159 (+ 78us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:25.247304 (+ 145us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:25.247396 (+ 92us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:25.248012 (+ 616us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:25.248374 (+ 362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0616 19:59:25.248421 (+ 47us) write_op.cc:312] APPLY: finished
0616 19:59:25.248646 (+ 225us) log.cc:844] Serialized 41 byte log entry
0616 19:59:25.248807 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:25.248897 (+ 90us) write_op.cc:454] Released schema lock
0616 19:59:25.248934 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:25.249270 (+ 336us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:25.249297 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4359,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":1908,"apply.run_cpu_time_us":1991,"apply.run_wall_time_us":2152,"num_ops":4,"prepare.queue_time_us":126944,"prepare.run_cpu_time_us":2887,"prepare.run_wall_time_us":4654,"replication_time_us":2831,"thread_start_us":54188,"threads_started":1}]]}
I20260616 19:59:28.242923 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1898) took 1052 ms. Trace:
W20260616 19:59:28.244874 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.243366 7968 rpcz_store.cc:276] 0616 19:59:27.190682 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:27.201148 (+ 10466us) service_pool.cc:224] Handling call
0616 19:59:27.218343 (+ 17195us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:27.218540 (+ 197us) raft_consensus.cc:1534] Early marking committed up to index 5
0616 19:59:27.732285 (+513745us) log.cc:844] Serialized 14 byte log entry
0616 19:59:28.242589 (+510304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:28.242643 (+ 54us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:28.242689 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:28.242759 (+ 70us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:28.242870 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7569,"prepare.queue_time_us":165262,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":11119,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":937130,"threads_started":2}
I20260616 19:59:28.246994 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2102) took 1027 ms. Trace:
I20260616 19:59:28.247956 7934 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.255713 7969 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
I20260616 19:59:28.269343 7968 raft_consensus.cc:1240] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260616 19:59:28.269586 7963 raft_consensus.cc:1275] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260616 19:59:28.273291 7934 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.247563 8045 rpcz_store.cc:276] 0616 19:59:27.219950 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:27.220080 (+ 130us) service_pool.cc:224] Handling call
0616 19:59:27.220358 (+ 278us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:27.220534 (+ 176us) raft_consensus.cc:1534] Early marking committed up to index 4
0616 19:59:27.755778 (+535244us) log.cc:844] Serialized 14 byte log entry
0616 19:59:28.246473 (+490695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:28.246529 (+ 56us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:28.246573 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:28.246812 (+ 239us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:28.246940 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36034,"prepare.queue_time_us":168636,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":181688,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":967897,"threads_started":3,"wal-append.queue_time_us":462857}
W20260616 19:59:28.273277 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.277177 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.278023 8045 raft_consensus.cc:1813] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Handling vote request from an unknown peer 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:28.279778 8045 raft_consensus.cc:2410] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260616 19:59:28.247051 11356 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:28.282156 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:28.286921 8011 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:28.287038 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.256542 11021 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:28.249277 11320 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:28.271068 11277 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.374828 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.375787 8116 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:28.377353 8045 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
W20260616 19:59:28.377625 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 90782.8us
I20260616 19:59:28.379869 8040 raft_consensus.cc:1240] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260616 19:59:28.323622 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.382542 11380 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.383298 11380 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:28.290855 11392 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:28.388230 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:28.300400 7934 leader_election.cc:304] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:28.389729 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 89463.8us
I20260616 19:59:28.391814 11138 raft_consensus.cc:2749] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260616 19:59:28.277648 11080 consensus_queue.cc:1059] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260616 19:59:28.300590 11147 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.277869 11322 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:28.334973 11373 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.292125 11021 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:28.280547 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:28.291815 11386 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:28.395056 11387 consensus_meta.cc:287] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.582s user 0.008s sys 0.001s
I20260616 19:59:28.407971 11387 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.300817 11145 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.410364 11387 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:28.410293 11145 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.399223 11080 raft_consensus.cc:3055] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260616 19:59:28.411515 11080 raft_consensus.cc:740] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:28.412881 11145 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:28.412552 11080 consensus_queue.cc:260] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.413302 11383 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.413630 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.413877 11080 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:28.414269 7961 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:28.397775 8786 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.775s (SENT) [suppressed 44 similar messages]
I20260616 19:59:28.419914 8116 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
W20260616 19:59:28.400573 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:28.421005 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.401110 11147 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.422441 11078 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:59:28.423221 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.423555 11147 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:28.423834 7963 raft_consensus.cc:2468] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 10.
W20260616 19:59:28.401057 7964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1876) took 2208 ms (client timeout 1654 ms). Trace:
W20260616 19:59:28.429883 7964 rpcz_store.cc:269] 0616 19:59:26.192267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:26.192358 (+ 91us) service_pool.cc:224] Handling call
0616 19:59:28.400815 (+2208457us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54002f2954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:28.400955 (+ 140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813675776}
I20260616 19:59:28.401412 11322 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.424692 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.278613 11297 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:28.425652 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.448956 7969 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 10.
W20260616 19:59:28.374223 11336 consensus_meta.cc:287] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 1.858s user 0.010s sys 0.003s
I20260616 19:59:28.432610 7968 raft_consensus.cc:1275] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260616 19:59:28.404254 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.459036 8114 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 7.
I20260616 19:59:28.406816 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.274644 11161 raft_consensus.cc:493] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.428985 8784 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
W20260616 19:59:28.283049 11379 consensus_meta.cc:287] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.636s user 0.003s sys 0.005s
I20260616 19:59:28.394510 7934 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:28.444139 8786 leader_election.cc:336] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 2.240s (SENT)
I20260616 19:59:28.461069 8786 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.403036 8811 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:28.407132 11386 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.413916 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:28.444770 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 65782.6us
I20260616 19:59:28.289533 10998 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:28.460249 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 322699us
I20260616 19:59:28.336174 11330 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:28.381003 11410 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:28.446360 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.430027 11380 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:28.402124 11373 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.406359 11263 consensus_queue.cc:1048] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260616 19:59:28.430225 11277 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:28.607980 8083 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s user 0.015s sys 0.032s
W20260616 19:59:28.646252 8083 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s user 0.016s sys 0.032s
W20260616 19:59:28.645816 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 251374us
I20260616 19:59:28.646704 8085 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.495014 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.402556 11021 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:28.645099 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.465734 7961 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:28.649823 8038 raft_consensus.cc:2410] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260616 19:59:28.647599 8110 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:28.570180 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.494025 8040 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9. Ops: []
W20260616 19:59:28.602164 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 158040us
I20260616 19:59:28.401774 11381 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:28.516350 8784 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:28.506258 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.647686 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:28.507716 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.657538 8085 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.528077 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.647790 8044 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
W20260616 19:59:28.656055 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 139827us
W20260616 19:59:28.660087 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:28.654966 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.662717 8036 raft_consensus.cc:1240] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:28.404137 11392 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260616 19:59:28.518014 11334 consensus_meta.cc:287] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.700s user 0.008s sys 0.001s
I20260616 19:59:28.670277 8119 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Advancing to term 12
W20260616 19:59:28.664506 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.566555 11297 raft_consensus.cc:2749] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:28.652370 11138 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.667776 11142 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:28.517827 8813 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 11. Current term is 12. Ops: []
I20260616 19:59:28.666438 8784 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
W20260616 19:59:28.666340 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.688241 11138 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.689970 8101 tablet_service.cc:1558] Processing DeleteTablet for tablet f24cd55ee50d4bcfaee34b2adaa9b08a with delete_type TABLET_DATA_TOMBSTONED (TS 6827184e3cef42c2ada15b2d31afee4b not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53906
I20260616 19:59:28.662576 11145 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:28.678146 11301 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.677721 11312 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.690776 11157 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:28.677429 11300 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.696816 11300 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.698979 7966 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.690943 11138 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.672734 11055 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.691140 11338 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260616 19:59:28.704173 11338 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:28.703778 11055 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.704638 11338 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:28.691169 11145 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:28.706627 11055 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:28.673955 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:28.707542 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.692276 11301 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.709647 11301 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.674347 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.710712 8040 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:28.692956 11312 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.712931 11312 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:28.717271 8043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2093) took 2349 ms. Trace:
I20260616 19:59:28.717715 8043 rpcz_store.cc:276] 0616 19:59:26.367566 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:26.520802 (+153236us) service_pool.cc:224] Handling call
0616 19:59:26.521424 (+ 622us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:26.521900 (+ 476us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:26.521944 (+ 44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:26.522606 (+ 662us) log.cc:844] Serialized 26 byte log entry
0616 19:59:26.558742 (+ 36136us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:26.558781 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:26.558847 (+ 66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:28.707916 (+2149069us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b24002a4208. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c085e94 00007fe86c083f9b 00007fe86c112d39 00007fe86c1129c5 00007fe86c111f20 00007fe86eeac828 00007fe86eeacf88 00007fe86ee84738 00007fe86ee7fb0a 00007fe86ee7ef56
0616 19:59:28.716965 (+ 9049us) raft_consensus.cc:1685] finished
0616 19:59:28.716992 (+ 27us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:28.717208 (+ 216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23030,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":3890,"spinlock_wait_cycles":62049536,"thread_start_us":35677,"threads_started":1,"wal-append.queue_time_us":87432}
I20260616 19:59:28.718756 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:28.703478 11246 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:28.465775 11336 raft_consensus.cc:2749] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:28.677692 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.677994 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.737483 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:28.517311 11409 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:28.680380 7968 raft_consensus.cc:1240] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:28.680955 8112 raft_consensus.cc:2410] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:59:28.653155 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:28.657600 10998 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.706856 11145 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:28.741343 11409 raft_consensus.cc:697] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:28.742288 10998 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:28.742532 11409 consensus_queue.cc:237] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.668387 11326 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:28.715173 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.675140 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:28.719990 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:28.721421 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.755532 8116 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:28.755895 8043 raft_consensus.cc:2410] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260616 19:59:28.663373 11410 raft_consensus.cc:2804] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:28.517506 8818 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 LEARNER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260616 19:59:28.669090 11080 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:28.678618 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:28.766844 7961 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:28.735630 8110 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:28.449985 11383 consensus_queue.cc:1059] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.222s
I20260616 19:59:28.653829 11210 consensus_queue.cc:1048] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260616 19:59:28.754458 7967 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
I20260616 19:59:28.774000 7970 raft_consensus.cc:1240] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:28.498186 11161 raft_consensus.cc:515] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:28.768667 8011 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:28.758414 7962 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:28.721449 11334 raft_consensus.cc:2749] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:28.778544 8011 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:28.651422 11379 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.801770 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:28.801801 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:28.802671 8114 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:28.797107 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.801239 8042 raft_consensus.cc:1275] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260616 19:59:28.808547 8042 raft_consensus.cc:1240] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:28.809656 8039 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:28.758853 11410 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:28.654278 11330 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.820436 8011 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:28.816834 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.821543 8044 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:28.817624 8041 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260616 19:59:28.766399 11080 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:28.828835 8118 raft_consensus.cc:1240] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:28.815960 8110 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:28.803303 11367 raft_consensus.cc:2804] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:28.834805 8114 raft_consensus.cc:1240] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 10. Current term is 11. Ops: []
I20260616 19:59:28.824458 8116 raft_consensus.cc:2410] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:59:28.814970 11383 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:28.839536 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.819028 11410 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:28.850780 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:28.832526 11080 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:28.803503 11320 consensus_queue.cc:1048] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260616 19:59:28.764679 11326 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:28.835243 11297 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
W20260616 19:59:28.858402 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:28.949451 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.869968 8085 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:28.834515 11367 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:29.047127 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:29.065577 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.071477 11240 raft_consensus.cc:3055] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:29.095938 11240 raft_consensus.cc:740] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:29.097028 11240 consensus_queue.cc:260] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.098753 11240 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:29.029202 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.027156 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:29.103718 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:29.109153 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:28.842314 11378 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:28.835909 11328 raft_consensus.cc:3055] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:28.852933 11383 raft_consensus.cc:697] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:29.119652 8044 raft_consensus.cc:2410] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260616 19:59:29.139403 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 269561us
I20260616 19:59:29.150607 8085 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:29.142830 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:29.152096 8110 raft_consensus.cc:3060] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:29.146124 8115 raft_consensus.cc:1275] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260616 19:59:29.093916 11367 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:29.089781 11373 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:29.089668 11420 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.161193 8115 raft_consensus.cc:1240] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:28.955363 11423 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:28.972496 11323 consensus_meta.cc:287] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.734s user 0.012s sys 0.000s
W20260616 19:59:29.163002 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.160082 8085 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:29.009804 11300 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.172777 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.179827 8084 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:29.137907 11393 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.148073 11302 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:29.112793 11322 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:29.217656 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:29.213225 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 336641us
I20260616 19:59:29.164021 11434 tablet_replica.cc:333] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: stopping tablet replica
I20260616 19:59:29.113876 11145 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:29.222707 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 62718.1us
W20260616 19:59:29.225239 7934 leader_election.cc:336] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 1.740s (SENT)
I20260616 19:59:29.223428 8042 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:29.162580 11378 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
W20260616 19:59:29.251715 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 72018.7us
I20260616 19:59:29.254233 8085 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:29.255573 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.256453 8042 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:29.257589 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.258284 8042 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:29.259307 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.171989 11328 raft_consensus.cc:740] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:29.222579 11422 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.170823 10998 consensus_queue.cc:1059] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260616 19:59:29.271817 8085 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:29.271775 8110 raft_consensus.cc:2468] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:29.274852 8042 raft_consensus.cc:1275] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:59:29.171038 11424 raft_consensus.cc:2749] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260616 19:59:29.270151 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 50341.5us
I20260616 19:59:29.226634 11210 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:29.174431 11386 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.180640 11426 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
W20260616 19:59:29.309321 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 84110.8us
W20260616 19:59:29.294646 11429 log.cc:927] Time spent T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.447s user 0.000s sys 0.001s
I20260616 19:59:29.263641 11378 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:29.223798 11393 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.216168 11423 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.222348 11055 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260616 19:59:29.270138 11161 leader_election.cc:290] T 2e660e68b6c54e9095967ae4b226b0d2 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.308388 8043 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260616 19:59:29.354809 11432 log.cc:927] Time spent T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.265s user 0.000s sys 0.001s
I20260616 19:59:29.355643 8043 raft_consensus.cc:1275] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260616 19:59:29.310003 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 655012us
I20260616 19:59:29.358927 11365 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.270838 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
W20260616 19:59:29.295078 8119 consensus_meta.cc:287] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.621s user 0.010s sys 0.000s
I20260616 19:59:29.311162 8085 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:29.265882 11322 raft_consensus.cc:2749] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:29.283924 11379 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:29.326573 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.280980 11422 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:29.384124 8811 consensus_meta.cc:287] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.715s user 0.004s sys 0.004s
I20260616 19:59:29.187673 11420 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.352281 11321 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader election won for term 11
W20260616 19:59:29.352735 11433 log.cc:927] Time spent T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.397s user 0.000s sys 0.001s
I20260616 19:59:29.353989 11326 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:29.385705 11157 consensus_meta.cc:287] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.691s user 0.010s sys 0.000s
I20260616 19:59:29.384553 10998 consensus_queue.cc:1059] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260616 19:59:29.376597 11381 consensus_queue.cc:1048] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
W20260616 19:59:29.389207 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 78133.8us
I20260616 19:59:29.389832 8118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 282) took 1099 ms. Trace:
I20260616 19:59:29.390337 11403 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:29.391140 11403 consensus_queue.cc:1059] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260616 19:59:29.391856 11403 consensus_queue.cc:1243] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer b022cac8cb5e46b7a39a748e81e257a7 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260616 19:59:29.368554 8119 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 12.
I20260616 19:59:29.381439 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:29.393069 8816 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
W20260616 19:59:29.369989 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 75203.7us
I20260616 19:59:29.393924 8084 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:29.381790 11142 consensus_meta.cc:287] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.689s user 0.008s sys 0.002s
I20260616 19:59:29.395000 8045 raft_consensus.cc:1275] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260616 19:59:29.395366 8045 pending_rounds.cc:85] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 4
I20260616 19:59:29.395651 8045 pending_rounds.cc:107] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260616 19:59:29.395896 8045 raft_consensus.cc:2889] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:29.396720 7933 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:29.396922 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:29.397706 11325 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:29.397745 8008 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:29.398257 11325 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.398720 11325 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:29.398989 11366 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.399490 11142 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:29.399667 11157 raft_consensus.cc:3055] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:29.392927 8085 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:29.398073 8084 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:29.401280 8036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2550) took 1074 ms. Trace:
I20260616 19:59:29.401754 8036 rpcz_store.cc:276] 0616 19:59:28.326464 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.328524 (+ 2060us) service_pool.cc:224] Handling call
0616 19:59:28.328951 (+ 427us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.668942 (+339991us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:29.400972 (+732030us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400292f54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:29.401210 (+ 238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767039744}
I20260616 19:59:29.343729 11328 consensus_queue.cc:260] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.391785 11423 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.382181 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:29.409017 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.385067 8043 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260616 19:59:29.390251 8118 rpcz_store.cc:276] 0616 19:59:28.290192 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.290455 (+ 263us) service_pool.cc:224] Handling call
0616 19:59:28.290834 (+ 379us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.829302 (+538468us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:29.389620 (+560318us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b540032aad4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:29.389775 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236844416}
I20260616 19:59:29.406286 8039 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:59:29.318699 11302 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:29.307276 11330 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:29.399503 11366 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.294068 11383 consensus_queue.cc:237] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.433005 11379 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260616 19:59:29.387778 11338 consensus_meta.cc:287] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.683s user 0.008s sys 0.002s
I20260616 19:59:29.400168 11157 raft_consensus.cc:740] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:29.390118 11381 consensus_queue.cc:1059] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
W20260616 19:59:29.441313 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 158007us
I20260616 19:59:29.386072 11422 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.388001 8811 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
W20260616 19:59:29.635185 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.622303 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 281) took 1339 ms. Trace:
I20260616 19:59:29.664294 8111 rpcz_store.cc:276] 0616 19:59:28.282690 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.282801 (+ 111us) service_pool.cc:224] Handling call
0616 19:59:28.333985 (+ 51184us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.841791 (+507806us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:28.841823 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:28.841871 (+ 48us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:28.841911 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:29.622053 (+780142us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b5400437954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:29.622100 (+ 47us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:29.622244 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167347072}
I20260616 19:59:29.666131 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:29.667729 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:29.567338 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:29.557335 11246 consensus_meta.cc:287] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.837s user 0.007s sys 0.003s
W20260616 19:59:29.555137 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:29.442860 7934 leader_election.cc:336] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 1.864s (SENT)
I20260616 19:59:29.508260 8040 raft_consensus.cc:1275] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260616 19:59:29.486907 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:29.673736 8040 pending_rounds.cc:85] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting all ops after (but not including) 3
I20260616 19:59:29.674008 8118 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:29.674146 8040 pending_rounds.cc:107] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260616 19:59:29.674675 8040 raft_consensus.cc:2889] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260616 19:59:29.496083 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.459980 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:29.657217 7935 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:29.638185 11334 raft_consensus.cc:3055] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Stepping down as leader of term 3
I20260616 19:59:29.676362 7935 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:29.552928 11420 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:29.573096 11145 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531)
I20260616 19:59:29.657887 11379 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:29.678754 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 235909us
I20260616 19:59:29.459615 11326 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.695631 8118 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
W20260616 19:59:29.487042 11445 log.cc:927] Time spent T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.167s user 0.000s sys 0.000s
W20260616 19:59:29.481643 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.670260 8116 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:29.701694 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:29.702446 8116 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:29.703414 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:29.568797 11021 consensus_meta.cc:287] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.906s user 0.009s sys 0.002s
I20260616 19:59:29.409008 11210 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260616 19:59:29.588841 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:29.559358 11442 log.cc:927] Time spent T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.099s user 0.000s sys 0.000s
I20260616 19:59:29.459419 11162 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.512473 11147 raft_consensus.cc:2804] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:29.629184 11409 raft_consensus.cc:3055] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 LEADER]: Stepping down as leader of term 3
W20260616 19:59:29.670543 8811 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52470 (request call id 164) took 1414 ms (client timeout 1588 ms). Trace:
I20260616 19:59:29.676961 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:29.582986 11408 log.cc:927] Time spent T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 1.503s user 0.003s sys 0.000s
I20260616 19:59:29.677277 7935 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:29.578284 11277 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:29.459419 11078 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260616 19:59:29.586236 11410 consensus_meta.cc:287] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.711s user 0.009s sys 0.000s
I20260616 19:59:29.438210 11423 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.678584 8113 raft_consensus.cc:2393] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 4: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
W20260616 19:59:29.566174 11080 consensus_meta.cc:287] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.713s user 0.004s sys 0.005s
W20260616 19:59:29.679880 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 320380us
I20260616 19:59:29.710007 11147 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.680127 8117 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 8.
I20260616 19:59:29.709797 11409 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
W20260616 19:59:29.686139 7934 leader_election.cc:336] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 1.944s (SENT)
I20260616 19:59:29.678371 11384 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:29.678439 11334 raft_consensus.cc:740] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:29.687443 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:29.730592 8816 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:29.728482 11384 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.730937 11334 consensus_queue.cc:260] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.732076 11334 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:29.719497 11158 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260616 19:59:29.709748 11162 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.734381 11366 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.688643 11157 consensus_queue.cc:260] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.736090 11162 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:29.736690 8112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 2245) took 2503 ms (client timeout 1969 ms). Trace:
I20260616 19:59:29.736838 11157 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:59:29.737115 8112 rpcz_store.cc:269] 0616 19:59:27.233678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:27.234680 (+ 1002us) service_pool.cc:224] Handling call
0616 19:59:29.736502 (+2501822us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400055d54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:29.736625 (+ 123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3288142208}
I20260616 19:59:29.738562 8112 raft_consensus.cc:1275] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260616 19:59:29.739647 8112 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:29.740734 7933 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:29.670164 11164 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:29.681349 11415 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:29.741899 8041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2558) took 1071 ms. Trace:
I20260616 19:59:29.741953 11415 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.742619 11415 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:29.562105 11367 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.746084 11367 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.668673 11302 raft_consensus.cc:697] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:29.747989 8038 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260616 19:59:29.748464 11302 consensus_queue.cc:237] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:29.711059 8811 rpcz_store.cc:269] 0616 19:59:28.255567 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.256654 (+ 1087us) service_pool.cc:224] Handling call
0616 19:59:29.669608 (+1412954us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:29.742422 8041 rpcz_store.cc:276] 0616 19:59:28.670630 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.670768 (+ 138us) service_pool.cc:224] Handling call
0616 19:59:28.671105 (+ 337us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.822564 (+151459us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:29.741670 (+919106us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400293e54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:29.741829 (+ 159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336125184}
I20260616 19:59:29.705288 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:29.753362 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 355321us
I20260616 19:59:29.557533 11393 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:29.684711 11379 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:29.711584 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:29.712654 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 319801us
I20260616 19:59:29.757874 8085 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:29.707217 11210 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.719225 8111 raft_consensus.cc:1275] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260616 19:59:29.671579 11325 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.723397 8786 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:29.511386 11328 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:29.459640 11321 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:29.718742 11336 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260616 19:59:29.785590 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 44928.9us
I20260616 19:59:29.718004 11319 consensus_queue.cc:1048] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260616 19:59:29.707659 11326 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.718954 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:29.708560 11021 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.798652 11321 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.799526 11021 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.724052 11423 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.801795 11423 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.794567 11277 consensus_queue.cc:1048] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260616 19:59:29.731460 11384 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:29.804728 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:29.806386 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:29.667274 11338 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.724957 11380 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.624022 11378 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:29.812666 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 86762.4us
I20260616 19:59:29.726040 11147 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:29.754318 8084 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:29.789655 7967 raft_consensus.cc:1275] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:59:29.793521 7964 raft_consensus.cc:1275] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260616 19:59:29.821952 8043 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260616 19:59:29.796651 8818 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:29.797271 11453 raft_consensus.cc:2804] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:29.760567 11210 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:29.792955 11336 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.829979 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.830570 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:29.762934 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 83070.9us
I20260616 19:59:29.822019 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:29.832036 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:29.793638 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 70324.7us
I20260616 19:59:29.725863 11080 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.830664 8043 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:29.722422 11410 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.779441 10969 consensus_queue.cc:1048] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260616 19:59:29.792721 11456 raft_consensus.cc:2764] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260616 19:59:29.861824 8043 raft_consensus.cc:1240] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:29.725289 11451 consensus_queue.cc:1048] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260616 19:59:29.848018 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:29.831866 11453 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.851027 11325 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:29.851824 11410 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:29.835191 10998 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:29.835022 11336 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:29.848251 11330 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:29.852806 8114 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.898s user 0.012s sys 0.046s
W20260616 19:59:29.877774 8114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s user 0.013s sys 0.046s
I20260616 19:59:29.795311 11450 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260616 19:59:29.855223 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 100977us
I20260616 19:59:29.855381 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:29.856580 8085 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:29.833834 7934 leader_election.cc:336] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 1.966s (SENT)
I20260616 19:59:29.812067 11021 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:29.863117 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:29.832654 11277 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:29.865963 8817 raft_consensus.cc:1240] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:29.867631 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.884132 8036 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:29.868053 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:29.885915 7964 raft_consensus.cc:2393] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 4: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:29.795496 11373 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260616 19:59:29.872852 8043 raft_consensus.cc:1275] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260616 19:59:29.854305 11338 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:29.835409 11158 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260616 19:59:29.828961 11380 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.881073 7934 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:29.887347 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:29.896075 8043 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
W20260616 19:59:29.887399 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.869095 11453 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:29.944681 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:29.945039 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.897644 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:29.945910 8040 raft_consensus.cc:2410] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:59:29.763121 11412 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
I20260616 19:59:29.878749 11450 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:29.878654 10998 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.869225 11409 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:29.899612 8116 raft_consensus.cc:1275] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260616 19:59:30.031399 8043 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:29.956321 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:30.072679 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:30.037978 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 153949us
I20260616 19:59:30.068728 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2272) took 1500 ms. Trace:
W20260616 19:59:29.911664 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:29.883208 11021 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:30.080125 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:29.924826 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 68675.8us
W20260616 19:59:30.016711 8009 connection.cc:256] RPC call timeout handler was delayed by -2.06419968900002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
I20260616 19:59:30.082896 8008 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:29.993333 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 100902us
W20260616 19:59:30.084833 8119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34110 (request call id 278) took 1803 ms (client timeout 1775 ms). Trace:
I20260616 19:59:30.085029 8115 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260616 19:59:30.085260 8119 rpcz_store.cc:269] 0616 19:59:28.281811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.281932 (+ 121us) service_pool.cc:224] Handling call
0616 19:59:30.084634 (+1802702us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400056254. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:30.084750 (+ 116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888599296}
I20260616 19:59:29.988216 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:30.086952 8111 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 12: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
W20260616 19:59:30.073851 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 240023us
I20260616 19:59:30.079202 8114 rpcz_store.cc:276] 0616 19:59:28.568270 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.835629 (+267359us) service_pool.cc:224] Handling call
0616 19:59:28.847007 (+ 11378us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.847205 (+ 198us) raft_consensus.cc:1534] Early marking committed up to index 7
0616 19:59:28.954102 (+106897us) log.cc:844] Serialized 14 byte log entry
0616 19:59:30.068372 (+1114270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:30.068426 (+ 54us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:30.068482 (+ 56us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.068543 (+ 61us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:30.068669 (+ 126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13604,"prepare.queue_time_us":96297,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":67717,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4971,"thread_start_us":1100367,"threads_started":2,"wal-append.queue_time_us":789103}
I20260616 19:59:30.092026 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:30.092121 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:30.094187 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.092504 8784 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:29.980098 11297 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:29.887136 11330 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.894819 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:29.894042 11277 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:29.946841 11346 consensus_queue.cc:1059] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.156s
W20260616 19:59:30.138489 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 239732us
I20260616 19:59:29.895344 11456 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:30.105400 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:30.145262 8110 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:30.069963 11450 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:29.967841 11162 consensus_queue.cc:1048] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.156s
I20260616 19:59:30.113102 7967 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:30.105579 8784 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
W20260616 19:59:30.102869 11460 log.cc:927] Time spent T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260616 19:59:30.134539 8037 raft_consensus.cc:2468] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:30.067898 11326 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.136945 8045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2551) took 1756 ms. Trace:
I20260616 19:59:30.163044 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.162652 8045 rpcz_store.cc:276] 0616 19:59:28.380675 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.381087 (+ 412us) service_pool.cc:224] Handling call
0616 19:59:28.381357 (+ 270us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:29.398407 (+1017050us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.136624 (+738217us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000e88d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:30.136865 (+ 241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2330312576}
I20260616 19:59:29.967954 11403 consensus_queue.cc:1048] T 38333e3e28e64678b975f1c136bcd790 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.147s
I20260616 19:59:30.097121 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:30.174561 11474 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:30.182147 11474 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:30.182462 7966 raft_consensus.cc:1275] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260616 19:59:30.153398 11409 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 3 FOLLOWER]: Advancing to term 4
I20260616 19:59:30.182725 11297 raft_consensus.cc:3055] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:30.183368 11297 raft_consensus.cc:740] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:30.184118 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.187206 11330 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:30.138497 8811 raft_consensus.cc:3060] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:30.097157 11458 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:29.979782 11145 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.148s
I20260616 19:59:30.102074 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:30.156461 8042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2114) took 1136 ms. Trace:
W20260616 19:59:30.158576 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 53094.1us
I20260616 19:59:30.103966 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:30.096611 11373 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.093271 11240 raft_consensus.cc:2764] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260616 19:59:30.165517 8119 raft_consensus.cc:2393] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:30.166386 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:29.978094 11302 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:30.129498 11164 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260616 19:59:30.173427 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.086638 11379 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:30.102814 11461 log.cc:927] Time spent T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.135s user 0.000s sys 0.000s
I20260616 19:59:30.183926 7966 raft_consensus.cc:1240] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:30.193543 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:30.193544 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:30.195497 8042 rpcz_store.cc:276] 0616 19:59:29.020106 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.302787 (+282681us) service_pool.cc:224] Handling call
0616 19:59:29.303594 (+ 807us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:29.400949 (+ 97355us) raft_consensus.cc:1534] Early marking committed up to index 8
0616 19:59:29.437812 (+ 36863us) log.cc:844] Serialized 14 byte log entry
0616 19:59:29.809397 (+371585us) log.cc:844] Serialized 14 byte log entry
0616 19:59:30.015229 (+205832us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b18001e6a60. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c1a7e1a 00007fe8758b35d9 00007fe86c0661f0 00007fe86c230b24 00007fe86c230a95 00007fe86f2dd440 00007fe86f2efaf6 00007fe86f300667 00007fe86f3005e8
0616 19:59:30.015410 (+ 181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:30.015466 (+ 56us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:30.015509 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.156050 (+140541us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b54003aaa54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:30.156130 (+ 80us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:30.156376 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201370,"prepare.queue_time_us":41073,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":8410,"raft.queue_time_us":229941,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":15799,"spinlock_wait_cycles":294778624,"thread_start_us":369539,"threads_started":1,"wal-append.queue_time_us":443007}
I20260616 19:59:30.243283 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:30.244778 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.244735 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.245968 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.246704 8042 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:30.214743 8111 raft_consensus.cc:1275] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260616 19:59:30.248554 8111 raft_consensus.cc:3055] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:30.248970 8111 raft_consensus.cc:740] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:59:30.191403 11454 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.323s user 0.000s sys 0.001s
I20260616 19:59:30.163853 11080 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.216763 8045 raft_consensus.cc:2468] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
W20260616 19:59:30.258728 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.226248 8037 raft_consensus.cc:2410] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260616 19:59:30.128683 11459 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.312s
I20260616 19:59:30.190471 11312 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.182505 11474 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:30.191793 11458 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:30.233337 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:30.238247 11379 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:30.265213 7967 raft_consensus.cc:2468] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:30.249946 8111 consensus_queue.cc:260] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.274533 8118 raft_consensus.cc:1275] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260616 19:59:30.278275 8118 pending_rounds.cc:85] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 2
I20260616 19:59:30.278568 8118 pending_rounds.cc:107] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260616 19:59:30.278847 8118 raft_consensus.cc:2889] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:30.274744 8037 raft_consensus.cc:1275] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260616 19:59:30.220193 11373 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:30.293889 11457 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.126s user 0.000s sys 0.001s
W20260616 19:59:30.300843 11468 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.155s user 0.000s sys 0.001s
I20260616 19:59:30.306564 8118 raft_consensus.cc:1275] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260616 19:59:30.192970 11470 consensus_queue.cc:1048] T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.165s
I20260616 19:59:30.217466 11158 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:30.271025 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:30.267277 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.223773 11297 consensus_queue.cc:260] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.271765 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:30.312359 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 216425us
W20260616 19:59:30.286572 11462 log.cc:927] Time spent T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.296s user 0.001s sys 0.000s
I20260616 19:59:30.314083 8045 raft_consensus.cc:3060] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:30.316145 7934 leader_election.cc:336] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): Timed out: RequestConsensusVote RPC to 127.6.159.131:40887 timed out after 2.017s (SENT)
I20260616 19:59:30.313655 8042 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:30.275511 11458 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:30.189340 11450 raft_consensus.cc:515] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.228891 11302 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.274256 11473 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:30.331938 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.272993 11312 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.337986 11473 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.351136 11473 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:30.333086 11328 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.352577 11328 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.357136 8110 raft_consensus.cc:1240] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:30.357580 8114 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:30.357594 7969 raft_consensus.cc:1813] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Handling vote request from an unknown peer 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:30.368767 7969 raft_consensus.cc:2393] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 6: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:59:30.369066 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 285863us
I20260616 19:59:30.369511 7934 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:30.359872 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.362470 8045 raft_consensus.cc:2468] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
W20260616 19:59:30.332652 11334 consensus_meta.cc:287] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.600s user 0.009s sys 0.001s
I20260616 19:59:30.320254 11158 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.332762 11412 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:30.372917 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.358054 11297 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:59:30.370555 8009 connection.cc:256] RPC call timeout handler was delayed by -1.9743183269999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260616 19:59:30.377293 8009 connection.cc:256] RPC call timeout handler was delayed by -1.9660996889999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
I20260616 19:59:30.366546 10998 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:30.379750 11476 log.cc:927] Time spent T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
I20260616 19:59:30.378912 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:30.405489 8110 raft_consensus.cc:2376] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9. Re-sending same reply.
I20260616 19:59:30.406379 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:30.377674 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.355973 11346 consensus_queue.cc:1048] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
W20260616 19:59:30.408478 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:30.411144 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:30.411527 8009 connection.cc:256] RPC call timeout handler was delayed by -1.7369694749999098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
W20260616 19:59:30.411990 8009 connection.cc:256] RPC call timeout handler was delayed by -1.6928694749999522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260616 19:59:30.412420 8009 connection.cc:256] RPC call timeout handler was delayed by -1.6203460819999691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.334s for a response
W20260616 19:59:30.412985 8009 connection.cc:256] RPC call timeout handler was delayed by -1.3927821040000481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2646s for a response
W20260616 19:59:30.413357 8009 connection.cc:256] RPC call timeout handler was delayed by -1.3787901759999386s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20640000000000003s for a response
I20260616 19:59:30.413595 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:30.413764 8009 connection.cc:256] RPC call timeout handler was delayed by -1.3426166399999602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
I20260616 19:59:30.414279 8110 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:59:30.414355 8009 connection.cc:256] RPC call timeout handler was delayed by -1.2914901760000248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
W20260616 19:59:30.414904 8009 connection.cc:256] RPC call timeout handler was delayed by -1.2563901759999681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260616 19:59:30.415295 8009 connection.cc:256] RPC call timeout handler was delayed by -1.2371597349999774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260616 19:59:30.415668 8009 connection.cc:256] RPC call timeout handler was delayed by -1.19969017599999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2263s for a response
I20260616 19:59:30.373411 11472 raft_consensus.cc:2749] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:30.417548 7934 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:30.416057 8009 connection.cc:256] RPC call timeout handler was delayed by -1.116716639999936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
I20260616 19:59:30.407104 11451 consensus_queue.cc:1048] T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260616 19:59:30.417088 11240 raft_consensus.cc:493] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:30.433058 11240 raft_consensus.cc:515] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.435204 11240 leader_election.cc:290] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.372500 11162 consensus_queue.cc:1048] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260616 19:59:30.375197 11412 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.411768 11386 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.460608 8084 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:30.462141 8084 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.472406 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:30.473294 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:30.474074 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.449514 11451 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:30.478053 8084 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:30.436753 11319 consensus_queue.cc:1059] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260616 19:59:30.528263 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 110815us
I20260616 19:59:30.530561 7934 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:30.531688 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.490873 8040 raft_consensus.cc:1240] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:30.479727 10998 consensus_queue.cc:1048] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260616 19:59:30.479939 11334 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:30.535185 7934 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.480041 11403 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:30.526849 10969 raft_consensus.cc:2804] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:30.534355 8110 raft_consensus.cc:2468] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:30.540879 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:30.477545 11321 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:30.470805 11386 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:30.551586 11479 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.139s user 0.001s sys 0.000s
I20260616 19:59:30.551105 11321 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:30.554103 11475 log.cc:927] Time spent T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.180s user 0.001s sys 0.000s
I20260616 19:59:30.552464 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:30.553521 11321 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:30.555717 11312 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.544826 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1908) took 2259 ms. Trace:
I20260616 19:59:30.544937 7934 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.536983 11334 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:30.563025 11334 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
W20260616 19:59:30.550140 11484 log.cc:927] Time spent T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.180s user 0.002s sys 0.000s
I20260616 19:59:30.555742 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.569618 8041 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:30.570554 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.571112 8041 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:30.571542 11434 raft_consensus.cc:2243] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260616 19:59:30.572098 8084 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:30.574620 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.575438 8041 raft_consensus.cc:2393] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:30.576248 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:30.555646 8110 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:30.477224 11470 raft_consensus.cc:2749] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:30.554952 11328 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.559271 7968 rpcz_store.cc:276] 0616 19:59:28.285242 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.458630 (+173388us) service_pool.cc:224] Handling call
0616 19:59:28.646321 (+187691us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.738777 (+ 92456us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.544609 (+1805832us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b54002085d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:30.544764 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77633408}
I20260616 19:59:30.560384 7934 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.480214 11456 consensus_queue.cc:1048] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260616 19:59:30.537060 11129 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:30.558784 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:30.565521 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:30.582316 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.582437 7966 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:30.582687 8040 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:30.583745 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:30.550539 8009 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:30.539479 11403 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:30.584815 11403 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:30.578564 8041 raft_consensus.cc:2393] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:30.588770 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:30.554617 8815 raft_consensus.cc:1275] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260616 19:59:30.590076 7966 raft_consensus.cc:2468] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 12.
I20260616 19:59:30.590517 8041 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:30.545819 11162 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:30.591039 11162 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:30.591178 7966 raft_consensus.cc:1240] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 3. Current term is 4. Ops: []
I20260616 19:59:30.591431 11162 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:30.592304 10998 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:30.592777 10998 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:30.593140 10998 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:30.566134 11158 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:30.589821 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:30.579543 8110 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260616 19:59:30.563388 10969 raft_consensus.cc:697] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:30.564337 11472 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:30.599381 8044 raft_consensus.cc:1240] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 8. Current term is 10. Ops: []
I20260616 19:59:30.601995 8040 raft_consensus.cc:1240] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 10. Current term is 11. Ops: []
I20260616 19:59:30.603699 7934 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:30.639786 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 89404.5us
W20260616 19:59:30.615944 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:30.582585 11336 consensus_meta.cc:287] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.699s user 0.010s sys 0.000s
I20260616 19:59:30.573024 11412 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.604624 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:30.567868 11386 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.648696 7966 raft_consensus.cc:1275] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260616 19:59:30.602885 11302 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:30.639789 11328 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260616 19:59:30.652812 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 49210.8us
I20260616 19:59:30.639348 11129 raft_consensus.cc:697] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:30.639042 11158 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:30.658350 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1899) took 3467 ms. Trace:
I20260616 19:59:30.658744 7965 rpcz_store.cc:276] 0616 19:59:27.191004 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:27.191311 (+ 307us) service_pool.cc:224] Handling call
0616 19:59:27.191565 (+ 254us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.228475 (+1036910us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.658082 (+2429607us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002fd354. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:30.658268 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2383357696}
W20260616 19:59:30.659694 8117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34110 (request call id 286) took 2255 ms (client timeout 1684 ms). Trace:
W20260616 19:59:30.660086 8117 rpcz_store.cc:269] 0616 19:59:28.404615 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.406574 (+ 1959us) service_pool.cc:224] Handling call
0616 19:59:30.659536 (+2252962us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001b62d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:30.659630 (+ 94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2805522688}
I20260616 19:59:30.661794 11450 leader_election.cc:290] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:30.663275 8116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2291) took 1259 ms. Trace:
W20260616 19:59:30.664777 11485 log.cc:927] Time spent T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.089s user 0.000s sys 0.000s
W20260616 19:59:30.667721 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:30.667930 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.669430 8115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2277) took 1851 ms. Trace:
I20260616 19:59:30.698994 8115 rpcz_store.cc:276] 0616 19:59:28.818294 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.175313 (+357019us) service_pool.cc:224] Handling call
0616 19:59:30.084757 (+909444us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:30.085555 (+ 798us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.669254 (+583699us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b5400056250. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:30.669373 (+ 119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2090688256}
W20260616 19:59:30.673393 11147 consensus_meta.cc:287] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.835s user 0.009s sys 0.001s
I20260616 19:59:30.676000 8039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2109) took 1831 ms. Trace:
I20260616 19:59:30.700796 8039 rpcz_store.cc:276] 0616 19:59:28.844790 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.844993 (+ 203us) service_pool.cc:224] Handling call
0616 19:59:29.403226 (+558233us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:29.415029 (+ 11803us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.675802 (+1260773us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400292f50. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:30.675941 (+ 139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678708608}
I20260616 19:59:30.701993 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.703015 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.677199 8038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2564) took 1880 ms. Trace:
W20260616 19:59:30.677588 11453 consensus_meta.cc:287] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.722s user 0.008s sys 0.002s
W20260616 19:59:30.678342 11210 consensus_meta.cc:287] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.830s user 0.006s sys 0.004s
I20260616 19:59:30.679777 11337 consensus_queue.cc:1048] T e8890d947e59455187876dccb01c8c06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:59:30.655016 11412 raft_consensus.cc:2749] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260616 19:59:30.685998 11487 consensus_queue.cc:1059] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260616 19:59:30.687218 7934 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.639621 11240 consensus_queue.cc:1048] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
W20260616 19:59:30.689574 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:30.706547 11384 consensus_meta.cc:287] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.893s user 0.005s sys 0.005s
I20260616 19:59:30.721062 7935 leader_election.cc:304] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:30.690980 11323 consensus_queue.cc:1048] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260616 19:59:30.692484 8041 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 12.
I20260616 19:59:30.724807 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:30.725399 8041 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:30.692652 11021 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:30.693392 11415 consensus_meta.cc:287] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.950s user 0.009s sys 0.003s
I20260616 19:59:30.696760 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:30.697547 8785 connection.cc:256] RPC call timeout handler was delayed by -1.1491609749998588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260616 19:59:30.663664 8116 rpcz_store.cc:276] 0616 19:59:29.404067 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.759657 (+355590us) service_pool.cc:224] Handling call
0616 19:59:29.759945 (+ 288us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:30.067960 (+308015us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.663052 (+595092us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54001a72d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:30.663218 (+ 166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320182144}
I20260616 19:59:30.703838 8038 rpcz_store.cc:276] 0616 19:59:28.796788 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.797858 (+ 1070us) service_pool.cc:224] Handling call
0616 19:59:28.817718 (+ 19860us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:29.748416 (+930698us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.676975 (+928559us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b54003acad4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:30.677145 (+ 170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138764672}
I20260616 19:59:30.706185 8039 raft_consensus.cc:2393] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 7: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:30.670096 10998 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.596062 8811 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:30.710057 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:30.746601 8040 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:30.690200 11129 consensus_queue.cc:237] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.696391 11434 raft_consensus.cc:2272] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260616 19:59:30.733525 8786 leader_election.cc:304] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:30.736763 8085 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.749132 8085 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.649789 11336 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.751327 8085 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.752763 8085 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:30.758641 8786 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:30.798534 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 45865.7us
I20260616 19:59:30.714162 10969 consensus_queue.cc:237] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.709157 11147 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.713481 11453 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.745819 11409 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.713380 11210 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:30.882030 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 161048us
I20260616 19:59:30.753860 11397 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:30.753947 11331 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260616 19:59:30.885567 7962 raft_consensus.cc:1240] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:30.756368 11471 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260616 19:59:30.742358 11258 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:30.888501 8040 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:30.804373 11302 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:30.892269 8009 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:30.806099 11393 consensus_queue.cc:1048] T e9e060ef9a144de99b92bcba80c31176 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260616 19:59:30.729518 11384 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.858803 11297 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.866569 11412 raft_consensus.cc:2804] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:30.900748 7935 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:30.901136 8112 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
W20260616 19:59:30.904590 8785 leader_election.cc:336] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.515s (SENT)
I20260616 19:59:30.904735 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1895) took 3715 ms. Trace:
I20260616 19:59:30.928244 7970 rpcz_store.cc:276] 0616 19:59:27.189606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:27.189724 (+ 118us) service_pool.cc:224] Handling call
0616 19:59:27.213644 (+ 23920us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:28.774574 (+1560930us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.904509 (+2129935us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400257954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:30.904677 (+ 168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3587878656}
I20260616 19:59:30.865154 11386 consensus_queue.cc:1059] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260616 19:59:30.905437 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:30.873374 8811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52474 (request call id 272) took 1003 ms. Trace:
I20260616 19:59:30.910952 8009 leader_election.cc:400] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:30.807293 11381 raft_consensus.cc:2804] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260616 19:59:30.807015 11312 consensus_queue.cc:1048] T 303be6b16595410aae7147c6026435ff P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260616 19:59:30.919328 8113 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:30.878661 11410 consensus_queue.cc:1059] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260616 19:59:30.933501 8820 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 12: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:30.745751 11415 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.873193 11145 consensus_queue.cc:1048] T 56c25ff952f14593ad9cdf3cf561a445 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260616 19:59:30.866710 11450 consensus_queue.cc:1048] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260616 19:59:30.884462 11470 consensus_queue.cc:1048] T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260616 19:59:30.948930 8009 leader_election.cc:304] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:30.921442 11328 raft_consensus.cc:2749] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:30.932579 11490 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.924867 11397 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.937304 11337 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:30.956607 11336 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:30.934098 8811 rpcz_store.cc:276] 0616 19:59:29.855794 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.861675 (+ 5881us) service_pool.cc:224] Handling call
0616 19:59:30.803979 (+942304us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:30.967468 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.898460 11409 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.895256 11403 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.933676 11471 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:30.956725 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2567) took 1166 ms. Trace:
I20260616 19:59:30.978483 7969 rpcz_store.cc:276] 0616 19:59:29.790390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.807619 (+ 17229us) service_pool.cc:224] Handling call
0616 19:59:30.956507 (+1148888us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54002f2954. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:30.956654 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210192768}
I20260616 19:59:30.942386 8818 raft_consensus.cc:1240] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 10. Current term is 11. Ops: []
I20260616 19:59:30.938923 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:30.933843 11277 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:30.973517 7961 raft_consensus.cc:1240] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:30.986316 11320 raft_consensus.cc:2749] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:30.987103 11322 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.008451 11409 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:31.008255 11322 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.009039 11409 raft_consensus.cc:2749] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:30.976282 11490 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.010104 11162 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:30.994779 11302 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.011291 11302 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.012854 11147 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.010505 11322 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:31.014678 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:31.030418 8117 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:30.918437 11258 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:30.969152 11337 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:31.043090 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:31.044927 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:31.057543 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:30.937793 11297 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.004477 11331 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.011951 11490 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.017341 11302 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.047998 11162 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:31.049629 11258 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:31.065181 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:31.065896 11328 consensus_queue.cc:1059] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:31.073256 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:31.075356 11378 leader_election.cc:290] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:31.076071 8818 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 11. Current term is 12. Ops: []
I20260616 19:59:31.076970 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:31.079860 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:31.082527 11488 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.081614 11158 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Committing config change with OpId 9.10: config changed from index 6 to 10, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:31.083087 11346 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.084466 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:31.087211 7934 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:31.088096 11078 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.089821 8820 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Committing config change with OpId 9.10: config changed from index 6 to 10, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:31.091502 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:31.194725 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 107577us
I20260616 19:59:31.195519 7934 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:31.092285 11379 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.093199 11240 raft_consensus.cc:3055] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:31.178294 8813 raft_consensus.cc:3060] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:31.102407 11373 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.104933 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1987) took 1255 ms. Trace:
I20260616 19:59:31.215126 8114 rpcz_store.cc:276] 0616 19:59:29.849195 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.163306 (+314111us) service_pool.cc:224] Handling call
0616 19:59:31.104651 (+941345us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400214654. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:31.104844 (+ 193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266073472}
I20260616 19:59:31.106796 8110 raft_consensus.cc:1240] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:31.107828 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2583) took 1256 ms. Trace:
I20260616 19:59:31.217361 7967 rpcz_store.cc:276] 0616 19:59:29.851608 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.897145 (+ 45537us) service_pool.cc:224] Handling call
0616 19:59:31.107614 (+1210469us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b540027b0d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:31.107763 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79690624}
I20260616 19:59:31.111580 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2569) took 1320 ms. Trace:
I20260616 19:59:31.218618 7961 rpcz_store.cc:276] 0616 19:59:29.791052 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.807549 (+ 16497us) service_pool.cc:224] Handling call
0616 19:59:29.831683 (+ 24134us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:31.004124 (+1172441us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.111272 (+107148us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002fd854. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:31.111505 (+ 233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2625431808}
I20260616 19:59:31.138710 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.110301 8112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2286) took 1820 ms. Trace:
I20260616 19:59:31.220474 8112 rpcz_store.cc:276] 0616 19:59:29.289848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.760451 (+470603us) service_pool.cc:224] Handling call
0616 19:59:29.760796 (+ 345us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:30.927483 (+1166687us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.109893 (+182410us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540032c3d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:31.110187 (+ 294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622099200}
I20260616 19:59:31.222256 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:31.223250 8112 raft_consensus.cc:2393] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 7: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:31.153978 8784 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:31.048167 11453 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:31.164175 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:30.975000 11397 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.174292 7970 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260616 19:59:31.182346 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:31.141690 11337 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:31.016079 11471 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:31.176471 11474 consensus_meta.cc:287] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Time spent flushing consensus metadata: real 0.888s user 0.012s sys 0.000s
I20260616 19:59:31.198947 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:31.044281 11412 raft_consensus.cc:697] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:31.203579 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 254713us
I20260616 19:59:31.099252 8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52474 (request call id 266) took 2592 ms. Trace:
W20260616 19:59:31.215502 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 456936us
I20260616 19:59:31.140594 8111 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:31.308389 8817 rpcz_store.cc:276] 0616 19:59:28.507172 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.507364 (+ 192us) service_pool.cc:224] Handling call
0616 19:59:28.709348 (+201984us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5800003998. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6a3e 000000000036f859 00007fe8742348e0 00007fe8742fd7c4 00007fe8742fd831 00007fe8742bf6c0 00007fe8742aa029 00007fe8742a9ad2 00007fe86d81ed34
0616 19:59:28.709512 (+ 164us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:29.893297 (+1183785us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.098883 (+1205586us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400204254. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:31.099135 (+ 252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2663638784}
I20260616 19:59:31.185969 10998 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:31.175356 11409 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260616 19:59:31.206404 11240 raft_consensus.cc:740] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:31.306239 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:31.206580 11162 raft_consensus.cc:2804] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:31.318537 8114 raft_consensus.cc:2468] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:31.310731 8009 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:31.320834 8009 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:31.311440 11356 consensus_queue.cc:1059] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260616 19:59:31.210139 11258 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.354533 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2587) took 1500 ms. Trace:
I20260616 19:59:31.187395 11488 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.142961 11331 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.306317 11210 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.369195 7968 rpcz_store.cc:276] 0616 19:59:29.853704 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.655167 (+801463us) service_pool.cc:224] Handling call
0616 19:59:30.695786 (+ 40619us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:30.710963 (+ 15177us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:30.711008 (+ 45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:30.711542 (+ 534us) log.cc:844] Serialized 26 byte log entry
0616 19:59:31.068596 (+357054us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b1800412360. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86ee8c85c 00007fe86c216c0b 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe86ee7e5ab 00007fe86ee840c3 00007fe86ee84471
0616 19:59:31.068737 (+ 141us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:31.324507 (+255770us) log.cc:844] Serialized 14 byte log entry
0616 19:59:31.325092 (+ 585us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.325165 (+ 73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:31.325202 (+ 37us) raft_consensus.cc:1685] finished
0616 19:59:31.354275 (+ 29073us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:31.354463 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61662,"raft.queue_time_us":24479,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4866,"spinlock_wait_cycles":26197760,"thread_start_us":373413,"threads_started":2,"wal-append.queue_time_us":190044}
I20260616 19:59:31.234351 11021 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260616 19:59:31.380640 8111 raft_consensus.cc:1275] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260616 19:59:31.386832 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:31.447198 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:31.479516 8111 pending_rounds.cc:85] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 4
I20260616 19:59:31.480304 8111 pending_rounds.cc:107] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260616 19:59:31.480723 8111 raft_consensus.cc:2889] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:31.482071 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2299) took 1929 ms. Trace:
I20260616 19:59:31.482582 8111 rpcz_store.cc:276] 0616 19:59:29.553122 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.248068 (+694946us) service_pool.cc:224] Handling call
0616 19:59:30.248361 (+ 293us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:31.095951 (+847590us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b58000125d0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f24148a 00007fe86f2c3827 00007fe86f2dc3c7 00007fe86f2ddbbe 00007fe86f2e7f29 00007fe86f2e87f6 00007fe86f2e516f 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34
0616 19:59:31.481755 (+385804us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.482004 (+ 249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606842240}
I20260616 19:59:31.484344 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.451062 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.367043 8116 raft_consensus.cc:2468] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:31.210116 11379 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.206882 11078 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.313580 11378 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.210456 11502 raft_consensus.cc:2749] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260616 19:59:31.470271 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:31.318611 11415 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:31.367777 11334 consensus_meta.cc:287] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.802s user 0.011s sys 0.000s
I20260616 19:59:31.359032 11453 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260616 19:59:31.389117 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 68372.9us
I20260616 19:59:31.355464 11300 raft_consensus.cc:2749] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:59:31.493865 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 198269us
I20260616 19:59:31.197384 11346 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.453841 11356 consensus_queue.cc:1243] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer 6827184e3cef42c2ada15b2d31afee4b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260616 19:59:31.388583 11384 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:31.486605 11492 log.cc:927] Time spent T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.287s user 0.001s sys 0.000s
I20260616 19:59:31.526031 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:31.560775 11379 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:31.531612 7935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s user 0.052s sys 0.127s
I20260616 19:59:31.450162 11381 raft_consensus.cc:697] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 LEADER]: Becoming Leader. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
W20260616 19:59:31.495070 11263 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260616 19:59:31.451999 11403 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:31.542109 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 388213us
W20260616 19:59:31.538748 8785 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.598s user 0.005s sys 0.008s
I20260616 19:59:31.318305 11495 consensus_queue.cc:1059] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.225s
I20260616 19:59:31.386219 11258 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.469370 11393 consensus_queue.cc:1059] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260616 19:59:31.328312 11474 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.539642 11323 raft_consensus.cc:493] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.604000 11453 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:31.559432 8115 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:31.563637 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.721711 8116 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
W20260616 19:59:31.576105 8085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s user 0.029s sys 0.154s
W20260616 19:59:31.722348 8085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s user 0.031s sys 0.157s
W20260616 19:59:31.553481 11410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s user 0.007s sys 0.007s
I20260616 19:59:31.591936 8111 raft_consensus.cc:2410] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:59:31.593364 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:31.598644 8009 leader_election.cc:400] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
W20260616 19:59:31.615931 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:31.542177 11240 consensus_queue.cc:260] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:31.621836 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:31.625978 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:31.635165 8110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 293) took 1540 ms. Trace:
I20260616 19:59:31.562402 11505 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:31.726822 11453 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:31.748744 8819 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:31.742357 8110 rpcz_store.cc:276] 0616 19:59:30.094871 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.600825 (+505954us) service_pool.cc:224] Handling call
0616 19:59:31.106541 (+505716us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:31.216827 (+110286us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.634988 (+418161us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b5400214650. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:31.635110 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1162328960}
I20260616 19:59:31.750943 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.752569 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.753234 8110 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:31.633558 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:31.755869 7967 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 11.
W20260616 19:59:31.678045 7935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s user 0.054s sys 0.128s
I20260616 19:59:31.634342 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:31.609666 11458 consensus_meta.cc:287] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 1.271s user 0.012s sys 0.000s
W20260616 19:59:31.682621 8785 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.742s user 0.005s sys 0.009s
I20260616 19:59:31.635126 8813 raft_consensus.cc:2468] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:31.688272 11258 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:31.627259 11503 log.cc:927] Time spent T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.309s user 0.001s sys 0.000s
I20260616 19:59:31.778455 8042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2145) took 1636 ms. Trace:
I20260616 19:59:31.778854 8042 rpcz_store.cc:276] 0616 19:59:30.141692 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.319243 (+177551us) service_pool.cc:224] Handling call
0616 19:59:30.319560 (+ 317us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:30.327920 (+ 8360us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:30.327966 (+ 46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:30.343591 (+ 15625us) log.cc:844] Serialized 26 byte log entry
0616 19:59:30.694399 (+350808us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b180036d720. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c216587 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe86ee7e5ab 00007fe86ee840c3 00007fe86ee84471 00007fe86f2a0def
0616 19:59:31.025989 (+331590us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:31.026049 (+ 60us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.026149 (+ 100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:31.778163 (+752014us) raft_consensus.cc:1685] finished
0616 19:59:31.778185 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:31.778395 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8423,"raft.queue_time_us":18280,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":3477,"spinlock_wait_cycles":26797824,"thread_start_us":331363,"threads_started":1,"wal-append.queue_time_us":829402}
I20260616 19:59:31.782012 11474 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:31.771251 11373 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:31.783187 11373 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.785318 11373 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:31.690964 11436 log.cc:927] Time spent T 87bfd6ef4d304dc9899bf6fbd1260a11 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 2.183s user 0.000s sys 0.006s
I20260616 19:59:31.785938 11381 consensus_queue.cc:237] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.596570 11162 raft_consensus.cc:697] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:31.787369 11162 consensus_queue.cc:237] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.765268 11379 raft_consensus.cc:493] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.789307 11488 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:31.788856 11379 raft_consensus.cc:515] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.789983 11240 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:31.791567 11379 leader_election.cc:290] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.694936 8784 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:31.793540 11331 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.694558 8113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2281) took 2580 ms. Trace:
W20260616 19:59:31.695274 8119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 1983) took 1857 ms (client timeout 1598 ms). Trace:
I20260616 19:59:31.697290 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:31.794618 8119 rpcz_store.cc:269] 0616 19:59:29.837388 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.091756 (+254368us) service_pool.cc:224] Handling call
0616 19:59:31.695094 (+1603338us) spinlock_profiling.cc:243] Waited 68.8 ms on lock 0x7b5400438854. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:31.695212 (+ 118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158350720}
I20260616 19:59:31.736812 11384 raft_consensus.cc:3055] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:31.546150 11344 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:31.625530 11437 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 2.136s user 0.005s sys 0.001s
I20260616 19:59:31.719424 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.548430 11412 consensus_queue.cc:237] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.614920 11078 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.558908 8112 raft_consensus.cc:2376] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 11. Re-sending same reply.
I20260616 19:59:31.720394 11409 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.637179 11434 ts_tablet_manager.cc:1916] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260616 19:59:31.568967 11378 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.597309 11334 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.799398 11347 raft_consensus.cc:3055] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:31.800069 11474 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260616 19:59:31.626261 8816 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:31.628757 11367 raft_consensus.cc:3055] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:31.649470 11210 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:31.626813 11499 log.cc:927] Time spent T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.085s user 0.000s sys 0.000s
I20260616 19:59:31.730068 8009 leader_election.cc:403] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260616 19:59:31.776192 11458 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.726286 11323 raft_consensus.cc:515] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.763270 11380 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:31.693411 11397 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:31.726698 11410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s user 0.010s sys 0.011s
I20260616 19:59:31.764817 7961 raft_consensus.cc:1275] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260616 19:59:31.767273 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 838592us
W20260616 19:59:31.882123 8785 leader_election.cc:336] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.775s (SENT)
I20260616 19:59:31.880044 8110 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:31.766832 8085 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:31.884929 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 118156us
W20260616 19:59:31.757284 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 831416us
I20260616 19:59:31.758314 7965 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 11.
I20260616 19:59:31.768527 8116 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:31.721949 11451 raft_consensus.cc:3055] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 LEADER]: Stepping down as leader of term 11
I20260616 19:59:31.722214 11509 raft_consensus.cc:493] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:31.856328 11498 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.691582 11471 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.788875 11473 consensus_queue.cc:1059] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
W20260616 19:59:31.793881 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 99020.4us
W20260616 19:59:31.635921 11321 consensus_meta.cc:287] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 1.080s user 0.008s sys 0.000s
W20260616 19:59:31.632611 11497 log.cc:927] Time spent T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.444s user 0.000s sys 0.001s
I20260616 19:59:31.794427 8113 rpcz_store.cc:276] 0616 19:59:29.114358 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.731945 (+617587us) service_pool.cc:224] Handling call
0616 19:59:29.732366 (+ 421us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:30.937089 (+1204723us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.694319 (+757230us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400215554. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:31.694493 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729182976}
I20260616 19:59:31.796636 7967 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12. Ops: []
I20260616 19:59:31.799377 11403 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.805343 8044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2149) took 1123 ms. Trace:
I20260616 19:59:31.788766 11373 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:31.551698 11328 raft_consensus.cc:2804] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:31.800644 11384 raft_consensus.cc:740] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:31.748533 11505 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:31.818853 11409 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.910938 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:31.926932 7935 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:31.865221 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 266796us
I20260616 19:59:31.917256 8044 rpcz_store.cc:276] 0616 19:59:30.682197 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.683874 (+ 1677us) service_pool.cc:224] Handling call
0616 19:59:30.684210 (+ 336us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:30.684548 (+ 338us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:30.684577 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:30.685017 (+ 440us) log.cc:844] Serialized 26 byte log entry
0616 19:59:30.908486 (+223469us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:30.908525 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.908593 (+ 68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:31.805147 (+896554us) raft_consensus.cc:1685] finished
0616 19:59:31.805169 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:31.805289 (+ 120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1020,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":223107,"threads_started":1,"wal-append.queue_time_us":489834}
I20260616 19:59:31.928467 8009 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:31.865782 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 55986us
I20260616 19:59:31.929458 7933 leader_election.cc:304] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:31.808144 11344 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.904714 11397 raft_consensus.cc:2804] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
W20260616 19:59:31.888849 8785 leader_election.cc:336] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051): Timed out: RequestConsensusVote RPC to 127.6.159.130:43051 timed out after 1.725s (SENT)
I20260616 19:59:31.829267 11347 raft_consensus.cc:740] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:31.898976 8085 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:31.855516 11367 raft_consensus.cc:740] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:59:31.940277 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 51478.6us
I20260616 19:59:31.864296 11519 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.906251 8118 raft_consensus.cc:1240] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 7. Ops: []
W20260616 19:59:31.907851 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:31.908365 11474 consensus_queue.cc:1059] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260616 19:59:31.809984 8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52458 (request call id 377) took 1064 ms. Trace:
I20260616 19:59:31.877063 11515 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 7. Current term is 8.
I20260616 19:59:31.896544 11380 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:31.889881 11451 raft_consensus.cc:740] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:31.865660 11210 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.928920 11129 raft_consensus.cc:2804] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:31.865079 11453 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.941027 8085 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:31.925761 11328 raft_consensus.cc:493] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:31.968477 11380 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.907908 11498 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.976459 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:31.976543 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:31.977161 11380 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.978760 11409 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:31.979233 11451 consensus_queue.cc:260] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:31.980105 8009 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:31.980412 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:31.981401 8009 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:31.982245 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:31.982714 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:31.943559 11312 raft_consensus.cc:2804] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:31.983968 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 358) took 1649 ms. Trace:
I20260616 19:59:31.936259 11505 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:31.908869 11321 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.987855 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:31.990466 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 49516.4us
I20260616 19:59:31.907334 11509 raft_consensus.cc:515] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.950305 11410 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:31.925022 11320 consensus_queue.cc:1059] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260616 19:59:31.908962 11379 consensus_queue.cc:1048] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260616 19:59:31.928733 11403 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:31.950058 11469 raft_consensus.cc:3055] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:31.989470 7966 rpcz_store.cc:276] 0616 19:59:30.334939 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.653011 (+318072us) service_pool.cc:224] Handling call
0616 19:59:30.653277 (+ 266us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:30.653608 (+ 331us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:30.653634 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:30.696251 (+ 42617us) log.cc:844] Serialized 26 byte log entry
0616 19:59:30.896835 (+200584us) raft_consensus.cc:1645] Marking committed up to 9
0616 19:59:30.897024 (+ 189us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:30.897173 (+ 149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:31.983771 (+1086598us) raft_consensus.cc:1685] finished
0616 19:59:31.983792 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:31.983914 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1750,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":23659,"thread_start_us":93695,"threads_started":1,"wal-append.queue_time_us":321301}
I20260616 19:59:31.983184 8813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:31.992900 7935 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:31.968917 11346 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:31.937893 11480 consensus_queue.cc:1048] T 541911a4cee341d8911dc307cd4a7b44 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260616 19:59:31.984591 11328 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:31.979324 11498 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:31.971678 11129 raft_consensus.cc:493] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:32.030321 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.003729 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:31.936091 11373 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } }
I20260616 19:59:31.990300 10969 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:32.016055 11451 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:32.001116 11157 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260616 19:59:31.990720 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:31.978760 11210 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:32.096665 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:32.097436 8812 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:31.984120 11397 raft_consensus.cc:697] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:32.056213 7965 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:31.953384 11519 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:32.110103 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:32.076804 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 95478.7us
I20260616 19:59:31.989145 11515 raft_consensus.cc:2749] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260616 19:59:32.090969 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 98187.5us
I20260616 19:59:32.050977 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:32.111222 8009 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:31.953152 8815 rpcz_store.cc:276] 0616 19:59:30.746076 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.756183 (+ 10107us) service_pool.cc:224] Handling call
0616 19:59:31.625854 (+869671us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800003998. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86c071941 00007fe86c1f66c1 00007fe86c1f6a3e 000000000036f859 00007fe8742348e0 00007fe8742fd7c4 00007fe8742fd831 00007fe8742ab7a0 00007fe8742ab084 00007fe86d823834 00007fe86d8237c2
0616 19:59:31.809019 (+183165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28808064}
I20260616 19:59:32.026585 8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:32.029886 11469 raft_consensus.cc:740] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:32.053920 11410 raft_consensus.cc:2749] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:32.076138 11129 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:32.006258 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.030674 11331 consensus_queue.cc:1059] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260616 19:59:32.145572 8818 raft_consensus.cc:2376] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4. Re-sending same reply.
I20260616 19:59:32.146275 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:32.146783 11325 raft_consensus.cc:3055] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:32.147223 8816 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 12: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
W20260616 19:59:32.147531 11513 log.cc:927] Time spent T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.238s user 0.000s sys 0.001s
I20260616 19:59:32.133162 11312 raft_consensus.cc:697] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:32.149295 8117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2317) took 1057 ms. Trace:
I20260616 19:59:32.182360 8117 rpcz_store.cc:276] 0616 19:59:31.091710 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.801802 (+710092us) service_pool.cc:224] Handling call
0616 19:59:31.802163 (+ 361us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.802586 (+ 423us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:31.802624 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:31.803169 (+ 545us) log.cc:844] Serialized 26 byte log entry
0616 19:59:31.989452 (+186283us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:31.989491 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:31.989571 (+ 80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.149065 (+159494us) raft_consensus.cc:1685] finished
0616 19:59:32.149087 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.149237 (+ 150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4080,"thread_start_us":133835,"threads_started":1,"wal-append.queue_time_us":334276}
I20260616 19:59:32.100813 11157 raft_consensus.cc:493] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:32.154423 8082 leader_election.cc:304] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:32.184198 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:32.184840 8117 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:31.990779 11384 consensus_queue.cc:260] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:32.168076 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:32.152843 11336 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.105s user 0.011s sys 0.023s
I20260616 19:59:32.148267 6782 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260616 19:59:32.177098 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.031006 11367 consensus_queue.cc:260] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.181167 8817 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12. Ops: []
I20260616 19:59:32.148717 8009 leader_election.cc:304] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.163010 11534 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:32.193537 8082 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:32.194438 11473 raft_consensus.cc:2749] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260616 19:59:32.199110 7967 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 4.
I20260616 19:59:32.130656 11521 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:59:32.223723 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 75080.4us
W20260616 19:59:32.202847 11336 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.155s user 0.011s sys 0.023s
I20260616 19:59:32.165247 11325 raft_consensus.cc:740] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:32.198796 8811 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:32.189817 11312 consensus_queue.cc:237] T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:32.202569 11337 consensus_meta.cc:287] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.884s user 0.012s sys 0.000s
I20260616 19:59:32.233661 8009 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.175523 11323 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.188046 11331 consensus_queue.cc:1243] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 log is divergent from this leader: its last log entry 9.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260616 19:59:32.188838 11157 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:32.210155 11240 raft_consensus.cc:2749] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:32.199508 11410 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:32.236420 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:32.202862 11334 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:32.264127 8009 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.222481 11378 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:32.268363 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:32.270464 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.255209 11210 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:32.285851 8009 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.292394 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.274418 11534 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:32.284790 11367 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:32.278380 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.285316 11210 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:32.287621 11498 raft_consensus.cc:493] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:32.264999 11337 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.240072 11344 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.291214 11321 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:32.255643 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.218775 11469 consensus_queue.cc:260] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.287422 11240 raft_consensus.cc:2749] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260616 19:59:32.280474 11410 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.296191 11453 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.339360 7964 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:32.340358 11337 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.261279 11397 consensus_queue.cc:237] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.354056 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2313) took 1263 ms. Trace:
I20260616 19:59:32.354516 8111 rpcz_store.cc:276] 0616 19:59:31.090795 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.727255 (+636460us) service_pool.cc:224] Handling call
0616 19:59:31.727659 (+ 404us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.728137 (+ 478us) raft_consensus.cc:1534] Early marking committed up to index 7
0616 19:59:31.930360 (+202223us) log.cc:844] Serialized 14 byte log entry
0616 19:59:32.162541 (+232181us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:32.175562 (+ 13021us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.175708 (+ 146us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:32.229425 (+ 53717us) log.cc:844] Serialized 14 byte log entry
0616 19:59:32.229621 (+ 196us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.229692 (+ 71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.353827 (+124135us) raft_consensus.cc:1685] finished
0616 19:59:32.353849 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.353997 (+ 148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85970,"prepare.queue_time_us":484578,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":12681,"raft.queue_time_us":27514,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":39774,"spinlock_wait_cycles":7294208,"thread_start_us":331995,"threads_started":2,"wal-append.queue_time_us":340357}
W20260616 19:59:32.317512 11524 log.cc:927] Time spent T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20260616 19:59:32.319038 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2318) took 1227 ms. Trace:
I20260616 19:59:32.306421 11210 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:32.310917 11498 raft_consensus.cc:515] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:32.290055 11378 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260616 19:59:32.305678 11325 consensus_queue.cc:260] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.385545 11325 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:32.388938 11522 log.cc:927] Time spent T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.228s user 0.001s sys 0.000s
W20260616 19:59:32.389763 11520 log.cc:927] Time spent T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.232s user 0.001s sys 0.001s
I20260616 19:59:32.390086 11458 leader_election.cc:290] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:32.391556 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2593) took 1246 ms. Trace:
I20260616 19:59:32.391920 7970 rpcz_store.cc:276] 0616 19:59:31.144994 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.635920 (+490926us) service_pool.cc:224] Handling call
0616 19:59:31.636382 (+ 462us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.636702 (+ 320us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:31.636728 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:31.637170 (+ 442us) log.cc:844] Serialized 26 byte log entry
0616 19:59:31.930800 (+293630us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:31.962253 (+ 31453us) log.cc:844] Serialized 14 byte log entry
0616 19:59:32.143026 (+180773us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.143290 (+ 264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.391330 (+248040us) raft_consensus.cc:1685] finished
0616 19:59:32.391361 (+ 31us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.391499 (+ 138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74143,"prepare.queue_time_us":31136,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2465,"raft.queue_time_us":138555,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":12327,"thread_start_us":399758,"threads_started":2,"wal-append.queue_time_us":456147}
I20260616 19:59:32.393939 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.335969 11410 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.308418 8812 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 12: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:32.357456 11472 raft_consensus.cc:3055] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:32.396385 11472 raft_consensus.cc:740] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:32.398525 11526 log.cc:927] Time spent T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.144s user 0.000s sys 0.001s
W20260616 19:59:32.401005 11541 log.cc:927] Time spent T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
I20260616 19:59:32.404084 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.405342 7961 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:32.312131 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.361749 8117 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:32.409265 8117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 299) took 1239 ms. Trace:
I20260616 19:59:32.409647 8117 rpcz_store.cc:276] 0616 19:59:31.169901 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.361273 (+1191372us) service_pool.cc:224] Handling call
0616 19:59:32.361589 (+ 316us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:32.408960 (+ 47371us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.409196 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:32.302884 11403 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:32.378937 8114 rpcz_store.cc:276] 0616 19:59:31.091928 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.818136 (+726208us) service_pool.cc:224] Handling call
0616 19:59:31.866243 (+ 48107us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.866492 (+ 249us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:31.866531 (+ 39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:31.867114 (+ 583us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.077628 (+210514us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:32.077685 (+ 57us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.077784 (+ 99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.318788 (+241004us) raft_consensus.cc:1685] finished
0616 19:59:32.318809 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.318977 (+ 168us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":152492,"threads_started":1,"wal-append.queue_time_us":290676}
I20260616 19:59:32.396000 11381 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:32.433923 11381 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:32.413370 8111 raft_consensus.cc:1240] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260616 19:59:32.361706 11157 raft_consensus.cc:515] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:32.397081 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.444180 7970 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:32.308794 11515 raft_consensus.cc:2749] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:32.431469 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:32.436045 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2327) took 1223 ms. Trace:
I20260616 19:59:32.339581 11505 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:32.404932 11531 log.cc:927] Time spent T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.137s user 0.000s sys 0.000s
I20260616 19:59:32.452800 8111 rpcz_store.cc:276] 0616 19:59:31.213094 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.412621 (+1199527us) service_pool.cc:224] Handling call
0616 19:59:32.413066 (+ 445us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:32.435693 (+ 22627us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.435975 (+ 282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:32.451354 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.477141 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:32.485255 8114 raft_consensus.cc:1240] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:32.435891 11509 leader_election.cc:290] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.537021 11384 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:32.539091 11528 log.cc:927] Time spent T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.236s user 0.000s sys 0.000s
I20260616 19:59:32.540833 8119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2316) took 1449 ms. Trace:
I20260616 19:59:32.486423 11240 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:32.541249 8119 rpcz_store.cc:276] 0616 19:59:31.091491 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.795701 (+704210us) service_pool.cc:224] Handling call
0616 19:59:31.798977 (+ 3276us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.804968 (+ 5991us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:31.805015 (+ 47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:31.811334 (+ 6319us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.163108 (+351774us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:32.197833 (+ 34725us) log.cc:844] Serialized 14 byte log entry
0616 19:59:32.326170 (+128337us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.326275 (+ 105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.540583 (+214308us) raft_consensus.cc:1685] finished
0616 19:59:32.540607 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.540761 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":894,"prepare.queue_time_us":55304,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":25782,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":13399,"thread_start_us":403683,"threads_started":2,"wal-append.queue_time_us":411782}
I20260616 19:59:32.540833 11328 raft_consensus.cc:515] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:32.580948 11530 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.283s user 0.001s sys 0.000s
I20260616 19:59:32.582906 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.583572 7967 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:32.586917 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:32.587504 8044 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:32.587765 7969 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260616 19:59:32.587965 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.588819 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:32.592659 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:32.592938 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:32.593281 8036 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:32.593497 8037 raft_consensus.cc:3060] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
W20260616 19:59:32.586939 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 301149us
I20260616 19:59:32.595930 8009 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.597142 8110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2322) took 1503 ms. Trace:
I20260616 19:59:32.597398 8009 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.598538 8009 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.597530 8110 rpcz_store.cc:276] 0616 19:59:31.093815 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.900276 (+806461us) service_pool.cc:224] Handling call
0616 19:59:31.900688 (+ 412us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.909647 (+ 8959us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:31.909674 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:31.928497 (+ 18823us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.180590 (+252093us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:32.596172 (+415582us) log.cc:844] Serialized 14 byte log entry
0616 19:59:32.596697 (+ 525us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.596912 (+ 215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.596964 (+ 52us) raft_consensus.cc:1685] finished
0616 19:59:32.596977 (+ 13us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.597090 (+ 113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58646,"prepare.queue_time_us":91522,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":39032,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":13751,"thread_start_us":516976,"threads_started":2,"wal-append.queue_time_us":385455}
I20260616 19:59:32.593523 8042 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Committing config change with OpId 9.10: config changed from index 6 to 10, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:32.604064 8044 raft_consensus.cc:2410] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260616 19:59:32.607828 8009 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:32.605772 8116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2323) took 1511 ms. Trace:
I20260616 19:59:32.608871 8009 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:32.611004 8043 raft_consensus.cc:1240] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 11. Current term is 12. Ops: []
I20260616 19:59:32.609061 8116 rpcz_store.cc:276] 0616 19:59:31.094051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.896851 (+802800us) service_pool.cc:224] Handling call
0616 19:59:31.897293 (+ 442us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.899751 (+ 2458us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:31.899795 (+ 44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:31.906861 (+ 7066us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.130174 (+223313us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:32.130213 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.130275 (+ 62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.605583 (+475308us) raft_consensus.cc:1685] finished
0616 19:59:32.605603 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.605718 (+ 115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7897,"raft.queue_time_us":9646,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":16608,"thread_start_us":180243,"threads_started":1,"wal-append.queue_time_us":280029}
I20260616 19:59:32.589848 8085 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:32.586211 11373 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:32.588523 11519 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:32.632090 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2597) took 1471 ms. Trace:
W20260616 19:59:32.623615 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.613061 11410 consensus_queue.cc:1059] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.013s
I20260616 19:59:32.627282 11434 ts_tablet_manager.cc:1929] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260616 19:59:32.611152 11544 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:32.594785 11453 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:32.613341 11378 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:32.665546 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.666338 8045 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:32.628603 8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52458 (request call id 382) took 1730 ms. Trace:
I20260616 19:59:32.630097 11210 raft_consensus.cc:515] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:32.631925 11367 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:32.594728 11469 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:32.667459 8819 rpcz_store.cc:276] 0616 19:59:30.898177 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.945133 (+ 46956us) service_pool.cc:224] Handling call
0616 19:59:32.628392 (+1683259us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b54005b13d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:32.628544 (+ 152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366743424}
I20260616 19:59:32.605506 11509 raft_consensus.cc:2804] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:32.605592 11515 raft_consensus.cc:2749] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260616 19:59:32.605417 11534 raft_consensus.cc:2804] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:32.642478 8037 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:32.637095 7968 rpcz_store.cc:276] 0616 19:59:31.160382 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.739574 (+579192us) service_pool.cc:224] Handling call
0616 19:59:31.739994 (+ 420us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:31.740510 (+ 516us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:31.740550 (+ 40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:31.755376 (+ 14826us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.095453 (+340077us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:32.122396 (+ 26943us) log.cc:844] Serialized 14 byte log entry
0616 19:59:32.631485 (+509089us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b18001dc620. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c1a7e1a 00007fe8758b35d9 00007fe86c0661f0 00007fe86c230b24 00007fe86c230a95 00007fe86f2dd440 00007fe86f2efaf6 00007fe86f300667 00007fe86f3005e8
0616 19:59:32.631665 (+ 180us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.631738 (+ 73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:32.631793 (+ 55us) raft_consensus.cc:1685] finished
0616 19:59:32.631888 (+ 95us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:32.632029 (+ 141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8004,"prepare.queue_time_us":34831,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":511051,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":29960,"spinlock_wait_cycles":38928896,"thread_start_us":266124,"threads_started":1,"wal-append.queue_time_us":359108}
I20260616 19:59:32.643342 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.648391 11540 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:32.649366 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:32.650539 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 60776.1us
I20260616 19:59:32.697324 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.657821 8036 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
I20260616 19:59:32.641745 11328 leader_election.cc:290] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:32.586105 11240 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.610277 10998 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260616 19:59:32.660956 11549 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:32.711869 11549 raft_consensus.cc:493] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:32.712358 11549 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:32.662321 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2565) took 2874 ms. Trace:
I20260616 19:59:32.663740 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.594628 11403 consensus_queue.cc:1048] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260616 19:59:32.668943 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.675447 11386 raft_consensus.cc:3055] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:32.674891 7865 catalog_manager.cc:5696] T 7323d4e3a76f4800925df28a0f090072 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 8 to 9, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 9 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: NON_VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260616 19:59:32.678339 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:32.680430 11367 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.632884 8815 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260616 19:59:32.698979 8085 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:32.716738 8041 raft_consensus.cc:1240] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260616 19:59:32.717255 8037 raft_consensus.cc:1240] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:32.717849 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.718605 8041 raft_consensus.cc:2393] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 8: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:32.718513 11367 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.720480 11386 raft_consensus.cc:740] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:32.721993 11386 consensus_queue.cc:260] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:32.723392 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.724658 8043 raft_consensus.cc:2468] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
W20260616 19:59:32.699676 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:32.726409 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.700021 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.698256 8044 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:32.665518 11378 raft_consensus.cc:493] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
W20260616 19:59:32.705307 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.732501 8036 raft_consensus.cc:1275] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260616 19:59:32.703825 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.723244 11386 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:32.728521 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:32.716265 7962 rpcz_store.cc:276] 0616 19:59:29.788229 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:29.788814 (+ 585us) service_pool.cc:224] Handling call
0616 19:59:29.813879 (+ 25065us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:30.900477 (+1086598us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.662000 (+1761523us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003feb54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:32.662234 (+ 234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2464238336}
I20260616 19:59:32.698704 8038 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:32.746091 8041 raft_consensus.cc:2393] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:32.740059 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:32.685287 11434 log.cc:1199] T f24cd55ee50d4bcfaee34b2adaa9b08a P 6827184e3cef42c2ada15b2d31afee4b: Deleting WAL directory at /tmp/dist-test-taskUwKsZi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1781639809409380-6782-0/minicluster-data/ts-2-root/wals/f24cd55ee50d4bcfaee34b2adaa9b08a
W20260616 19:59:32.744331 11548 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260616 19:59:32.745405 8085 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:32.700472 11263 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:32.728331 11451 consensus_meta.cc:287] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.619s user 0.006s sys 0.004s
I20260616 19:59:32.737783 8036 raft_consensus.cc:1240] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
W20260616 19:59:32.704217 11129 consensus_meta.cc:287] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.535s user 0.007s sys 0.003s
I20260616 19:59:32.694865 11534 raft_consensus.cc:493] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:32.751510 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.684872 11509 raft_consensus.cc:493] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:32.731225 7865 catalog_manager.cc:5696] T 38333e3e28e64678b975f1c136bcd790 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 1 to 3, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 3 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:59:32.752854 11458 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:32.752961 11297 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:32.752785 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.755519 11258 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:32.755724 8085 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:32.769870 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:32.770946 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.695364 11498 leader_election.cc:290] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:32.772886 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:32.773489 11336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s user 0.030s sys 0.030s
I20260616 19:59:32.784014 7962 raft_consensus.cc:2468] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 4.
W20260616 19:59:32.779619 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.779313 7865 catalog_manager.cc:5696] T 5c4d4bfa06e04d2d8de1b13263001b1b P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 7 to 10, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 10 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:32.861729 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:32.861931 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 106259us
I20260616 19:59:32.862416 7962 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:32.862823 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:32.863590 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.682986 11453 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:32.743697 11367 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:32.863871 8820 raft_consensus.cc:2468] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:32.781373 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:32.865056 8817 raft_consensus.cc:2393] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 11: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:32.739481 11378 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:32.862394 7865 catalog_manager.cc:5696] T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:32.869831 7865 catalog_manager.cc:5696] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 5, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:59:32.874092 7865 catalog_manager.cc:5696] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 7 to 8, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:59:32.876806 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:32.877631 7962 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:32.878487 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:32.879060 7962 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:32.878315 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.879643 8041 raft_consensus.cc:2468] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:32.881913 7865 catalog_manager.cc:5696] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 7 to 8, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:32.781140 11458 raft_consensus.cc:493] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:32.704731 11540 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:32.806134 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 69125us
I20260616 19:59:32.752928 11505 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:32.932907 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:32.889855 7865 catalog_manager.cc:5696] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:32.940323 8038 raft_consensus.cc:2410] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260616 19:59:32.938241 8011 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:32.777141 11534 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:32.774442 11263 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.942291 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:32.947992 8043 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
W20260616 19:59:32.940846 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 79977.4us
I20260616 19:59:32.782702 11509 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:32.867503 11258 raft_consensus.cc:493] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:32.945003 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 63577.2us
I20260616 19:59:32.774677 11129 raft_consensus.cc:515] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:32.873054 11334 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:32.945972 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:32.934269 11521 consensus_meta.cc:287] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.690s user 0.007s sys 0.003s
I20260616 19:59:32.873236 11373 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:32.943051 7865 catalog_manager.cc:5696] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:33.037539 7865 catalog_manager.cc:5696] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 5, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 5 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:33.039561 7865 catalog_manager.cc:5696] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260616 19:59:32.958999 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:32.934796 11458 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:32.945967 11544 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:33.020108 7962 raft_consensus.cc:2468] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:33.048398 11544 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:33.048858 11544 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:33.049067 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.049731 7962 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:33.050571 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.051316 11210 leader_election.cc:290] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:33.004469 11553 log.cc:927] Time spent T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260616 19:59:33.030483 8043 raft_consensus.cc:2410] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260616 19:59:33.050227 11347 consensus_queue.cc:260] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.021147 8113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2325) took 1926 ms. Trace:
I20260616 19:59:33.052713 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:33.055286 8113 rpcz_store.cc:276] 0616 19:59:31.094523 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.948926 (+854403us) service_pool.cc:224] Handling call
0616 19:59:31.978239 (+ 29313us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:32.113767 (+135528us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:32.113811 (+ 44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:32.133910 (+ 20099us) log.cc:844] Serialized 564 byte log entry
0616 19:59:32.234427 (+100517us) raft_consensus.cc:1645] Marking committed up to 9
0616 19:59:32.234489 (+ 62us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:33.007141 (+772652us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002cb854. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:33.007190 (+ 49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:33.007234 (+ 44us) raft_consensus.cc:1685] finished
0616 19:59:33.007248 (+ 14us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:33.021064 (+ 13816us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310567296,"thread_start_us":100141,"threads_started":1,"wal-append.queue_time_us":218363}
I20260616 19:59:33.041529 7865 catalog_manager.cc:5696] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 2 to 3, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 3 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:32.876935 11367 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.056860 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.060911 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.064383 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.065866 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:33.067560 7968 raft_consensus.cc:2393] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 10: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:33.028126 11258 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:33.067983 7865 catalog_manager.cc:5696] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 reported cstate change: term changed from 4 to 6, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:33.071211 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.036283 11373 raft_consensus.cc:493] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:33.038080 11334 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:33.076056 8116 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:33.078138 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.078418 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:33.079185 7962 raft_consensus.cc:3060] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Advancing to term 12
W20260616 19:59:33.079979 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 192258us
I20260616 19:59:33.041504 11521 raft_consensus.cc:2749] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:33.080174 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.082213 8011 leader_election.cc:304] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:33.085547 8113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30950534b1834f6886db15f85f0abf14"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:33.085857 11540 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.087400 11210 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:33.088450 10998 raft_consensus.cc:493] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:33.089154 11373 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.088053 11210 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.089005 10998 raft_consensus.cc:515] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.092193 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.092831 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.093550 8011 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:33.093958 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.096654 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34110 (request call id 300) took 1286 ms. Trace:
I20260616 19:59:33.097335 8112 raft_consensus.cc:2393] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 10: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:33.097041 8114 rpcz_store.cc:276] 0616 19:59:31.810420 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.496503 (+686083us) service_pool.cc:224] Handling call
0616 19:59:32.551889 (+ 55386us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:32.552383 (+ 494us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:32.552426 (+ 43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:32.586112 (+ 33686us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.671146 (+ 85034us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:32.671189 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.671255 (+ 66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:33.096443 (+425188us) raft_consensus.cc:1685] finished
0616 19:59:33.096465 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:33.096597 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":253,"raft.queue_time_us":283,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":30723,"threads_started":1,"wal-append.queue_time_us":63446}
I20260616 19:59:33.112592 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:33.113113 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:33.115012 11561 log.cc:927] Time spent T 2e660e68b6c54e9095967ae4b226b0d2 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.061s user 0.000s sys 0.001s
I20260616 19:59:33.103062 11557 raft_consensus.cc:2804] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election won for term 8
W20260616 19:59:33.118965 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.095187 11458 raft_consensus.cc:515] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.082613 11334 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:33.081292 11434 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53906 (request call id 83) took 4374 ms. Trace:
I20260616 19:59:33.127202 8043 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:33.093369 11347 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:33.129281 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:33.132592 8041 raft_consensus.cc:2393] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:33.067956 11403 consensus_queue.cc:1059] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260616 19:59:33.133788 11403 raft_consensus.cc:3055] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:33.134160 11403 raft_consensus.cc:740] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:33.130033 11557 raft_consensus.cc:697] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:33.134999 11403 consensus_queue.cc:260] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.135529 11367 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.102989 11558 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:33.128715 11451 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:33.133085 11384 consensus_meta.cc:287] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7: Time spent flushing consensus metadata: real 0.595s user 0.008s sys 0.008s
I20260616 19:59:33.155627 7966 raft_consensus.cc:2410] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260616 19:59:33.157198 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:33.158736 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.147089 11558 raft_consensus.cc:493] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:33.153652 11403 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:33.175302 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.154454 11240 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.178136 8041 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:33.177332 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.173578 11558 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:33.190527 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.191897 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.192582 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.193110 7962 raft_consensus.cc:2468] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 12.
I20260616 19:59:33.194166 8114 raft_consensus.cc:2393] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:33.195113 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.196024 7962 raft_consensus.cc:2410] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260616 19:59:33.198940 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.182099 11263 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.159729 11451 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.202517 8043 raft_consensus.cc:2410] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260616 19:59:33.202631 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.188400 11210 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.207777 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.209581 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.135643 11557 consensus_queue.cc:237] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.195056 11129 leader_election.cc:290] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.211112 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.214085 8116 raft_consensus.cc:2368] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
W20260616 19:59:33.283310 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.295673 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.294320 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.274941 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.295096 8118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2303) took 3740 ms. Trace:
I20260616 19:59:33.281329 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.297956 8082 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:33.308512 8116 raft_consensus.cc:2368] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:33.263720 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.266456 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:33.244599 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.307756 8119 raft_consensus.cc:2468] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
W20260616 19:59:33.221036 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.308523 8118 rpcz_store.cc:276] 0616 19:59:29.554264 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:30.308407 (+754143us) service_pool.cc:224] Handling call
0616 19:59:30.308745 (+ 338us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:31.944699 (+1635954us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:33.294781 (+1350082us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540032cdd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:33.295001 (+ 220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3673403264}
I20260616 19:59:33.311976 8009 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:33.218743 8043 raft_consensus.cc:1275] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260616 19:59:33.309517 8041 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:33.313613 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.130257 11434 rpcz_store.cc:276] 0616 19:59:28.689121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:28.689668 (+ 547us) service_pool.cc:224] Handling call
0616 19:59:29.028478 (+338810us) ts_tablet_manager.cc:1151] Deleting tablet f24cd55ee50d4bcfaee34b2adaa9b08a
0616 19:59:29.100629 (+ 72151us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0616 19:59:32.362369 (+3261740us) tablet_metadata.cc:665] Metadata flushed
0616 19:59:32.627139 (+264770us) tablet_metadata.cc:665] Metadata flushed
0616 19:59:33.063588 (+436449us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1209274,"tablet-delete.queue_time_us":252989,"thread_start_us":102542,"threads_started":1}
I20260616 19:59:33.314260 8043 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:33.315274 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.216959 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.316910 8043 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:33.318955 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.273752 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12fbb557a1a4c488d787ecb4e750fad" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.205331 11458 leader_election.cc:290] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:33.278131 11549 consensus_meta.cc:287] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.565s user 0.006s sys 0.004s
I20260616 19:59:33.336057 8009 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:33.378414 8118 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 12.
I20260616 19:59:33.379616 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.379912 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:33.383023 8819 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 8: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:33.383378 11263 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:33.384043 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.380934 8085 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:33.379356 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:33.385874 8038 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 3.
I20260616 19:59:33.386436 8820 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 9
I20260616 19:59:33.387666 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.388103 8041 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
W20260616 19:59:33.267016 11556 log.cc:927] Time spent T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.129s user 0.000s sys 0.001s
I20260616 19:59:33.388882 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1f7e6b1ac4325b6bc7a648cbba327" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.388936 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.389483 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.384402 11263 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:33.390039 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 72320.6us
I20260616 19:59:33.278542 11451 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.393076 8119 raft_consensus.cc:2393] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:33.225193 11453 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.314661 11379 raft_consensus.cc:493] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:33.364436 11469 consensus_meta.cc:287] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b: Time spent flushing consensus metadata: real 0.683s user 0.009s sys 0.002s
I20260616 19:59:33.313119 10998 leader_election.cc:290] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.396469 7965 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:33.363178 11515 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:33.398257 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:33.279000 11240 raft_consensus.cc:493] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:33.399003 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.397410 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:33.399675 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8890d947e59455187876dccb01c8c06" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.407601 8820 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:33.362428 11384 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:33.317906 11565 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Leader election won for term 4
I20260616 19:59:33.408097 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:33.409258 8038 raft_consensus.cc:2410] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260616 19:59:33.317143 11367 consensus_queue.cc:1059] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260616 19:59:33.412041 8111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53918 (request call id 2331) took 1669 ms. Trace:
I20260616 19:59:33.412454 8111 rpcz_store.cc:276] 0616 19:59:31.743041 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.461440 (+718399us) service_pool.cc:224] Handling call
0616 19:59:32.461870 (+ 430us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:32.462302 (+ 432us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:32.462344 (+ 42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:32.602907 (+140563us) log.cc:844] Serialized 26 byte log entry
0616 19:59:32.877732 (+274825us) raft_consensus.cc:1645] Marking committed up to 4
0616 19:59:32.877773 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:32.877840 (+ 67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:33.411831 (+533991us) raft_consensus.cc:1685] finished
0616 19:59:33.411856 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:33.411989 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11785,"raft.queue_time_us":24043,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":4728,"thread_start_us":209297,"threads_started":1,"wal-append.queue_time_us":427715}
I20260616 19:59:33.413372 8119 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 10.
I20260616 19:59:33.278939 11544 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.394235 11563 raft_consensus.cc:2804] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader election won for term 12
I20260616 19:59:33.361856 11504 consensus_queue.cc:1059] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260616 19:59:33.417068 8042 raft_consensus.cc:1240] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:33.411818 11569 consensus_queue.cc:1059] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260616 19:59:33.320243 11210 consensus_queue.cc:1059] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260616 19:59:33.411645 7853 catalog_manager.cc:5027] TS 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887): tablet f24cd55ee50d4bcfaee34b2adaa9b08a (table test-workload [id=274d51639c084ed39f0f2badc5040b90]) successfully deleted
I20260616 19:59:33.420610 8785 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:33.418228 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.392262 11373 raft_consensus.cc:515] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.422463 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.426710 8041 raft_consensus.cc:2468] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 10.
W20260616 19:59:33.428666 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.391357 11549 raft_consensus.cc:515] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.424437 11381 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:33.431473 11378 consensus_meta.cc:287] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.547s user 0.005s sys 0.004s
I20260616 19:59:33.432024 11545 raft_consensus.cc:3055] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 LEADER]: Stepping down as leader of term 11
I20260616 19:59:33.431872 11378 raft_consensus.cc:515] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.432435 11545 raft_consensus.cc:740] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:33.433053 8036 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:33.417824 11565 raft_consensus.cc:697] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:33.423666 11515 raft_consensus.cc:2749] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:33.446448 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.448040 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:33.448784 8036 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.449501 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.450305 8038 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:33.458101 8036 raft_consensus.cc:2468] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:33.414395 11240 raft_consensus.cc:515] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.410071 11379 raft_consensus.cc:515] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.463397 8111 raft_consensus.cc:1240] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 5. Current term is 7. Ops: []
W20260616 19:59:33.479122 7934 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.414875 11384 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.427623 11563 raft_consensus.cc:697] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [term 12 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:59:33.463891 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.461988 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:33.489425 8041 raft_consensus.cc:2468] T 541911a4cee341d8911dc307cd4a7b44 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:33.466790 8036 raft_consensus.cc:1240] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 8. Current term is 9. Ops: []
I20260616 19:59:33.439845 11210 consensus_queue.cc:1243] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260616 19:59:33.486475 7934 connection.cc:256] RPC call timeout handler was delayed by -1.3192092549999188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260616 19:59:33.464838 11504 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260616 19:59:33.492995 7934 connection.cc:256] RPC call timeout handler was delayed by -1.2004092549999541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
I20260616 19:59:33.488545 11346 raft_consensus.cc:493] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
W20260616 19:59:33.497305 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 161353us
I20260616 19:59:33.441473 11258 raft_consensus.cc:515] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.504704 10998 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:33.504395 11346 raft_consensus.cc:515] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.520530 11334 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.521682 8112 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:33.522317 11346 leader_election.cc:290] T 2bf8afc79e3c417c814ac2fb57557096 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.508612 8118 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.523198 11379 leader_election.cc:290] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.524176 11544 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.525605 11240 leader_election.cc:290] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.527176 8113 raft_consensus.cc:606] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:33.527650 8113 raft_consensus.cc:614] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:33.511901 8114 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:33.520531 10998 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.518649 11263 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:33.539362 8115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53918 (request call id 2310) took 2449 ms (client timeout 2041 ms). Trace:
W20260616 19:59:33.553196 8115 rpcz_store.cc:269] 0616 19:59:31.090050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.091269 (+ 1219us) service_pool.cc:224] Handling call
0616 19:59:33.539107 (+2447838us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54001a6dd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:33.539269 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700636032}
I20260616 19:59:33.555207 8115 raft_consensus.cc:1240] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260616 19:59:33.526391 11565 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.544521 8009 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:33.542946 8044 raft_consensus.cc:1275] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260616 19:59:33.485256 11487 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:33.560770 8044 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260616 19:59:33.563604 8110 raft_consensus.cc:2393] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 6: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:33.559248 11240 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:33.566072 8044 raft_consensus.cc:3055] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:33.566569 8044 raft_consensus.cc:740] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:33.568325 8044 consensus_queue.cc:260] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.569643 8044 raft_consensus.cc:3060] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.573097 11512 raft_consensus.cc:3055] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:33.573516 11512 raft_consensus.cc:740] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:33.573683 10998 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.574361 11512 consensus_queue.cc:260] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.575601 11512 raft_consensus.cc:3060] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.576453 11334 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.577284 11563 consensus_queue.cc:237] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.579241 8044 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:59:33.581840 8044 raft_consensus.cc:1240] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:33.565841 11240 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.588544 8118 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
W20260616 19:59:33.595745 8786 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.610s (SENT) [suppressed 28 similar messages]
W20260616 19:59:33.596112 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 175555us
I20260616 19:59:33.597496 8785 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:33.599637 11549 leader_election.cc:290] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.600138 11240 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.601620 11373 leader_election.cc:290] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.603166 11258 leader_election.cc:290] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:33.606621 11534 consensus_meta.cc:287] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.576s user 0.004s sys 0.004s
W20260616 19:59:33.607019 11386 consensus_meta.cc:287] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.855s user 0.004s sys 0.004s
I20260616 19:59:33.607136 11534 raft_consensus.cc:515] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.609452 11534 leader_election.cc:290] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.613814 11545 consensus_queue.cc:260] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:33.616554 11509 consensus_meta.cc:287] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19: Time spent flushing consensus metadata: real 0.576s user 0.008s sys 0.003s
I20260616 19:59:33.617096 11509 raft_consensus.cc:515] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.619557 8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52458 (request call id 395) took 1424 ms. Trace:
I20260616 19:59:33.619997 8815 rpcz_store.cc:276] 0616 19:59:32.195331 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.195555 (+ 224us) service_pool.cc:224] Handling call
0616 19:59:32.632480 (+436925us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:32.723850 (+ 91370us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:33.619352 (+895502us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54005b13d0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:33.619497 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003655040}
I20260616 19:59:33.620669 11378 leader_election.cc:290] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.588878 11487 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.622879 11453 raft_consensus.cc:3055] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:33.623380 11453 raft_consensus.cc:740] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:33.623723 11487 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.619704 11509 leader_election.cc:290] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.627421 11532 raft_consensus.cc:2749] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:33.624856 11569 consensus_queue.cc:1048] T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260616 19:59:33.633522 11453 consensus_queue.cc:260] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.641131 11453 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.677875 7934 leader_election.cc:304] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:33.681360 7934 leader_election.cc:304] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:33.687937 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.689209 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2606) took 1651 ms. Trace:
W20260616 19:59:33.689214 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 144635us
I20260616 19:59:33.690759 8040 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 6.
I20260616 19:59:33.693444 8038 raft_consensus.cc:1240] T cdd9ba5e3e5c4292b330704225571321 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:33.695401 8039 raft_consensus.cc:2393] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:33.632036 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:33.704345 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7c5954a3dd147cea894b8e392ab2bd6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:33.706610 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:33.707535 8819 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 12.
I20260616 19:59:33.712903 11240 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:33.689942 7970 rpcz_store.cc:276] 0616 19:59:32.037938 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.587213 (+549275us) service_pool.cc:224] Handling call
0616 19:59:32.587580 (+ 367us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:32.663032 (+ 75452us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:32.663061 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:33.295900 (+632839us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b2c000328a8. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86dbdec71 00007fe86dbee5d2 00007fe86f320de1 00007fe86f2e581a 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22
0616 19:59:33.296435 (+ 535us) log.cc:844] Serialized 26 byte log entry
0616 19:59:33.418929 (+122494us) raft_consensus.cc:1645] Marking committed up to 7
0616 19:59:33.687541 (+268612us) log.cc:844] Serialized 14 byte log entry
0616 19:59:33.688139 (+ 598us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:33.688339 (+ 200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:33.688518 (+ 179us) raft_consensus.cc:1685] finished
0616 19:59:33.688678 (+ 160us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:33.688958 (+ 280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127898,"prepare.queue_time_us":87532,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":75492,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":17873,"spinlock_wait_cycles":44695680,"thread_start_us":381897,"threads_started":2,"wal-append.queue_time_us":169216}
I20260616 19:59:33.711915 8009 leader_election.cc:304] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:33.723739 11347 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:33.716490 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:33.724864 11347 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.787032 7970 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:33.646999 11558 raft_consensus.cc:515] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.790750 8045 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:33.787062 7934 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:33.792727 11558 leader_election.cc:290] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:33.795760 11258 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:33.796317 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.796653 11373 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:33.797070 7966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2604) took 1801 ms (client timeout 1792 ms). Trace:
W20260616 19:59:33.797704 7966 rpcz_store.cc:269] 0616 19:59:31.995659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.030067 (+ 34408us) service_pool.cc:224] Handling call
0616 19:59:33.796529 (+1766462us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003bced4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:33.796822 (+ 293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528690432}
I20260616 19:59:33.799120 11384 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.800307 11346 raft_consensus.cc:2804] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:33.800915 11346 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:33.796712 11258 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.803664 8037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51870 (request call id 2584) took 1108 ms. Trace:
I20260616 19:59:33.799814 11347 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.801230 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:33.816754 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.791772 11240 raft_consensus.cc:2749] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:33.805176 8037 rpcz_store.cc:276] 0616 19:59:32.695061 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.695420 (+ 359us) service_pool.cc:224] Handling call
0616 19:59:32.695726 (+ 306us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:32.717641 (+ 21915us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:33.803023 (+1085382us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5400293454. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:33.803414 (+ 391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49157888}
I20260616 19:59:33.829442 11157 leader_election.cc:290] T 3a82eaaad5b1444b9ca242cd0aaeed47 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 12 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:33.804993 11258 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:33.805740 11346 raft_consensus.cc:3060] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.841377 11344 raft_consensus.cc:2749] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260616 19:59:33.842921 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.843750 11451 consensus_queue.cc:1048] T 5ffc56f0158c4f8a99ad2d5a819bde34 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260616 19:59:33.851085 7970 raft_consensus.cc:2410] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260616 19:59:33.853518 7965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2610) took 1743 ms (client timeout 2203 ms). Trace:
W20260616 19:59:33.858402 7965 rpcz_store.cc:269] 0616 19:59:32.109715 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.109840 (+ 125us) service_pool.cc:224] Handling call
0616 19:59:33.852889 (+1743049us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540004c4d4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:33.853220 (+ 331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2956856064}
W20260616 19:59:33.856564 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.857051 11367 raft_consensus.cc:2749] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260616 19:59:33.860111 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.864513 7966 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
I20260616 19:59:33.865958 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.866277 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.868325 11569 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:33.869222 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.867472 7933 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:33.872272 11451 raft_consensus.cc:2804] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:33.872835 11451 raft_consensus.cc:493] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:33.873378 11451 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:33.870201 7966 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:33.797784 11373 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.839468 8009 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:33.856583 8786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 260695us
I20260616 19:59:33.883195 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.886049 7965 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 5.
W20260616 19:59:33.887660 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.889624 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.890815 7970 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 4.
I20260616 19:59:33.891049 8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:33.897354 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:33.891563 11381 raft_consensus.cc:3055] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260616 19:59:33.894393 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:33.894819 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:33.875937 11347 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:33.899770 8819 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 4.
I20260616 19:59:33.862044 8085 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:33.869063 11569 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:33.903987 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:33.903563 8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:33.905932 8820 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 12.
I20260616 19:59:33.898578 11381 raft_consensus.cc:740] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:33.911038 11410 raft_consensus.cc:2749] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260616 19:59:33.912420 7933 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:33.913007 11554 raft_consensus.cc:3055] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:33.913589 11554 raft_consensus.cc:740] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:33.915026 11554 consensus_queue.cc:260] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:33.911320 11575 consensus_queue.cc:1059] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260616 19:59:33.917534 11575 raft_consensus.cc:3055] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:33.917965 11575 raft_consensus.cc:740] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c06f6f0858fe4ce6bd5b55efeb5ba2d8, State: Running, Role: LEADER
I20260616 19:59:33.918463 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.924108 11554 raft_consensus.cc:3060] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:33.924512 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.925426 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1926) took 1289 ms. Trace:
I20260616 19:59:33.921487 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1917) took 1287 ms. Trace:
I20260616 19:59:33.933339 7933 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:33.925864 7968 rpcz_store.cc:276] 0616 19:59:32.635899 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.751224 (+115325us) service_pool.cc:224] Handling call
0616 19:59:33.925031 (+1173807us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540016ba54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:33.925240 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666497280}
I20260616 19:59:33.905397 8816 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:33.932760 7961 rpcz_store.cc:276] 0616 19:59:32.633809 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:33.917945 (+1284136us) service_pool.cc:224] Handling call
0616 19:59:33.921425 (+ 3480us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:33.945189 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 83578.7us
I20260616 19:59:33.925446 7967 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
W20260616 19:59:33.981060 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 47787.6us
W20260616 19:59:33.974251 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:33.978884 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.001642 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:34.008103 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 226260us
W20260616 19:59:33.980461 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 217946us
I20260616 19:59:34.010679 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1936) took 1361 ms. Trace:
I20260616 19:59:34.012584 7934 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:34.013404 8084 connection.cc:256] RPC call timeout handler was delayed by -1.3362121229999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
I20260616 19:59:34.011068 7967 rpcz_store.cc:276] 0616 19:59:32.643230 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:33.923947 (+1280717us) service_pool.cc:224] Handling call
0616 19:59:34.004740 (+ 80793us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:34.014180 8084 connection.cc:256] RPC call timeout handler was delayed by -1.2012121230000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260616 19:59:34.015162 8084 connection.cc:256] RPC call timeout handler was delayed by -1.197612122999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260616 19:59:34.015496 8084 connection.cc:256] RPC call timeout handler was delayed by -1.140912123000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
W20260616 19:59:34.015681 8084 connection.cc:256] RPC call timeout handler was delayed by -1.1166121229999817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260616 19:59:34.018290 11588 raft_consensus.cc:2749] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:33.938766 11575 consensus_queue.cc:260] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.020128 11410 raft_consensus.cc:493] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:34.021024 11410 raft_consensus.cc:515] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.025013 11575 raft_consensus.cc:3060] T f7c5954a3dd147cea894b8e392ab2bd6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:34.027776 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.029667 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.031217 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:33.934175 11347 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260616 19:59:33.978899 11258 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:33.984791 11574 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:34.041707 7970 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 12: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:33.981604 11539 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260616 19:59:34.009655 11373 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.040808 11258 raft_consensus.cc:493] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.040763 11347 raft_consensus.cc:493] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.045034 11574 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.054165 7933 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:34.054219 8036 raft_consensus.cc:1240] T f52f3e8cd74b471e9c8b0fc90839f7b0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:34.067802 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.055016 11258 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:34.048689 11539 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.065239 11588 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:34.043588 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.060117 11347 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:34.048095 11410 leader_election.cc:290] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.053717 11157 consensus_queue.cc:1059] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260616 19:59:34.074774 8009 leader_election.cc:400] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:59:34.087157 8009 leader_election.cc:403] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260616 19:59:34.079680 11500 raft_consensus.cc:3055] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Stepping down as leader of term 10
W20260616 19:59:34.092761 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 73953us
I20260616 19:59:34.075168 11539 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Advancing to term 13
I20260616 19:59:34.074936 11588 raft_consensus.cc:493] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.097254 11588 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:34.042101 11381 consensus_queue.cc:260] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.091188 11500 raft_consensus.cc:740] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.114641 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.112327 11410 raft_consensus.cc:3060] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:34.121690 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1922) took 1486 ms. Trace:
I20260616 19:59:34.122272 7963 rpcz_store.cc:276] 0616 19:59:32.635021 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.063352 (+1428331us) service_pool.cc:224] Handling call
0616 19:59:34.121593 (+ 58241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.127017 11587 raft_consensus.cc:493] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:34.127815 11587 raft_consensus.cc:515] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.128384 7934 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:34.122033 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
W20260616 19:59:34.129407 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.125455 8084 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.131997 8812 raft_consensus.cc:2410] T 43378ff312f94246811895976a8715d6 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260616 19:59:34.133760 11569 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.141378 8817 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:34.140441 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.144279 11055 raft_consensus.cc:2764] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260616 19:59:34.120319 11346 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.148485 11157 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:34.148934 11157 raft_consensus.cc:493] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.149326 11157 raft_consensus.cc:3060] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:34.149694 8044 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260616 19:59:34.150815 11586 log.cc:927] Time spent T f24cd55ee50d4bcfaee34b2adaa9b08a P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.144s user 0.001s sys 0.000s
I20260616 19:59:34.151226 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.147476 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24cd55ee50d4bcfaee34b2adaa9b08a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:34.152611 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 110633us
I20260616 19:59:34.153708 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1939) took 1072 ms. Trace:
I20260616 19:59:34.154367 11500 consensus_queue.cc:260] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.154942 11472 consensus_queue.cc:260] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.159974 11545 raft_consensus.cc:3060] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Advancing to term 12
I20260616 19:59:34.164389 11574 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.172125 11331 consensus_queue.cc:1059] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:34.173759 11469 raft_consensus.cc:3055] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Stepping down as leader of term 7
W20260616 19:59:34.173947 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.174297 11469 raft_consensus.cc:740] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.173972 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.181139 7970 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 8.
I20260616 19:59:34.187036 11539 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.154142 7966 rpcz_store.cc:276] 0616 19:59:33.080985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.150849 (+1069864us) service_pool.cc:224] Handling call
0616 19:59:34.153641 (+ 2792us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.153749 8009 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.220669 7961 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 13: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:34.220744 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.222820 8036 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:34.223690 11539 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 13 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.216590 11344 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:34.224321 7963 raft_consensus.cc:3060] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:34.224815 11344 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.227097 11344 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.175391 11469 consensus_queue.cc:260] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.228533 11469 raft_consensus.cc:3060] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:34.230751 8084 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:34.228422 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.233273 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:34.234236 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.234591 11346 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.234980 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.235761 7966 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:59:34.236339 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 82728.7us
I20260616 19:59:34.237201 8009 leader_election.cc:400] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10.
I20260616 19:59:34.233270 11380 raft_consensus.cc:2804] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:34.237125 11505 raft_consensus.cc:2804] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:34.237775 11380 raft_consensus.cc:493] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election (detected failure of leader c06f6f0858fe4ce6bd5b55efeb5ba2d8)
I20260616 19:59:34.237979 11505 raft_consensus.cc:493] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.238277 11380 raft_consensus.cc:3060] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:34.238485 11505 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
W20260616 19:59:34.242846 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.244479 11500 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:34.244390 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.245292 8039 raft_consensus.cc:3060] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:34.250833 11157 raft_consensus.cc:515] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.251626 8814 raft_consensus.cc:2468] T fd3a274b9cd54eea9bdf95a6e2c9e13a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:34.254809 11544 raft_consensus.cc:493] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:34.255584 7969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2617) took 1945 ms. Trace:
I20260616 19:59:34.274636 11544 raft_consensus.cc:515] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.274501 7969 rpcz_store.cc:276] 0616 19:59:32.185163 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.589661 (+404498us) service_pool.cc:224] Handling call
0616 19:59:32.589891 (+ 230us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:32.590082 (+ 191us) raft_consensus.cc:1534] Early marking committed up to index 6
0616 19:59:32.595068 (+ 4986us) log.cc:844] Serialized 14 byte log entry
0616 19:59:34.001331 (+1406263us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18000468a0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe8750c08f1 00007fe8750cd0c1 00007fe86c216587 00007fe875035759 00007fe86c232cdb 00007fe86c230f7f 00007fe86c233242 00007fe86ee7e5ab 00007fe86ee840c3 00007fe86ee845da 00007fe86f2efe0a
0616 19:59:34.107484 (+106153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:34.112087 (+ 4603us) raft_consensus.cc:1645] Marking committed up to 6
0616 19:59:34.119145 (+ 7058us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.123469 (+ 4324us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:34.130405 (+ 6936us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4617,"prepare.queue_time_us":4833,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2461,"raft.queue_time_us":5596,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":23059840,"thread_start_us":65841,"threads_started":1,"wal-append.queue_time_us":1518684}
I20260616 19:59:34.277097 11544 leader_election.cc:290] T 5ab514ead9d34ff598a454bd28675d12 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.277271 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.232553 8084 leader_election.cc:304] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.260860 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.278555 8036 raft_consensus.cc:2410] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260616 19:59:34.262066 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.263327 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:34.284339 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 51991.9us
I20260616 19:59:34.264905 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.285135 8042 raft_consensus.cc:2468] T eb7747b7f8144d689a26ea1780989278 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:34.286191 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:34.286473 11581 log.cc:927] Time spent T 6ea1f7e6b1ac4325b6bc7a648cbba327 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.493s user 0.004s sys 0.000s
I20260616 19:59:34.287232 8042 raft_consensus.cc:2393] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:34.284740 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.264493 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:34.266201 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 43872.1us
I20260616 19:59:34.288631 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1942) took 1194 ms. Trace:
I20260616 19:59:34.289052 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.289870 8042 raft_consensus.cc:2468] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 12.
I20260616 19:59:34.289942 8085 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:34.290665 8040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51876 (request call id 2169) took 1628 ms (client timeout 1555 ms). Trace:
I20260616 19:59:34.291056 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.291321 11565 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260616 19:59:34.291754 11565 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260616 19:59:34.265533 11593 log.cc:927] Time spent T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.147s user 0.000s sys 0.001s
W20260616 19:59:34.267864 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 142275us
I20260616 19:59:34.238413 8009 leader_election.cc:304] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.279048 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.279227 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.279954 8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.280105 11473 raft_consensus.cc:2749] T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:34.261132 11451 raft_consensus.cc:515] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.282754 8039 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:34.284550 8041 raft_consensus.cc:2410] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260616 19:59:34.291963 8042 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 9: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:34.267371 11594 log.cc:927] Time spent T 33c17f5e21da4e9893d22581a43ceafb P c06f6f0858fe4ce6bd5b55efeb5ba2d8: Append to log took a long time: real 0.123s user 0.000s sys 0.000s
I20260616 19:59:34.289006 7968 rpcz_store.cc:276] 0616 19:59:33.094477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.284401 (+1189924us) service_pool.cc:224] Handling call
0616 19:59:34.288568 (+ 4167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.287312 8084 leader_election.cc:304] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:34.300599 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.309057 7963 raft_consensus.cc:2468] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:34.292091 11565 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Advancing to term 9
W20260616 19:59:34.291093 8040 rpcz_store.cc:269] 0616 19:59:32.662327 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:32.663427 (+ 1100us) service_pool.cc:224] Handling call
0616 19:59:34.290434 (+1627007us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54001faed4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:34.290572 (+ 138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2221969152}
I20260616 19:59:34.294095 11410 raft_consensus.cc:2749] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:59:34.314358 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.314641 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.313900 11397 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:34.317467 8044 raft_consensus.cc:2468] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:34.307070 8816 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Advancing to term 13
I20260616 19:59:34.318398 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.319031 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.301285 11380 raft_consensus.cc:515] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.322746 8039 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:34.322988 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.323808 7970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1932) took 1683 ms. Trace:
I20260616 19:59:34.324129 7970 rpcz_store.cc:276] 0616 19:59:32.640535 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.308609 (+1668074us) service_pool.cc:224] Handling call
0616 19:59:34.323751 (+ 15142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.325115 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9837c086539a4592bb5aada5872652ca" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.325696 7970 raft_consensus.cc:3060] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:34.327116 7934 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:34.327364 11380 leader_election.cc:290] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.327777 11473 raft_consensus.cc:2749] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:34.326839 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.328651 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.328541 11397 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.331400 7963 raft_consensus.cc:2410] T 2e660e68b6c54e9095967ae4b226b0d2 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260616 19:59:34.331815 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49190 (request call id 1918) took 1697 ms. Trace:
I20260616 19:59:34.328709 7968 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:34.332093 7963 rpcz_store.cc:276] 0616 19:59:32.634085 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.317974 (+1683889us) service_pool.cc:224] Handling call
0616 19:59:34.331756 (+ 13782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.332785 11397 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.333562 11576 raft_consensus.cc:493] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader de4c5ce3ea8345ae9068f825b8d60e19)
I20260616 19:59:34.333839 7966 raft_consensus.cc:2468] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:34.334949 8084 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.335179 8038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51876 (request call id 2171) took 1025 ms. Trace:
I20260616 19:59:34.335525 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2621) took 1687 ms. Trace:
I20260616 19:59:34.335638 8038 rpcz_store.cc:276] 0616 19:59:33.309465 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:33.451069 (+141604us) service_pool.cc:224] Handling call
0616 19:59:33.451399 (+ 330us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:33.694062 (+242663us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.334888 (+640826us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400011654. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e61e1 00007fe86f2e4c1e 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076
0616 19:59:34.335119 (+ 231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555087232}
I20260616 19:59:34.336696 11380 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:34.336863 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.334091 11576 raft_consensus.cc:515] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.337622 7963 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
W20260616 19:59:34.338701 11589 log.cc:927] Time spent T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.305s user 0.002s sys 0.000s
I20260616 19:59:34.339310 11576 leader_election.cc:290] T 6c06fb426fd349ac813aef7f33c47e85 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.334856 8039 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:34.337244 8084 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.340556 8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.336021 7966 rpcz_store.cc:276] 0616 19:59:32.651554 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.234618 (+1583064us) service_pool.cc:224] Handling call
0616 19:59:34.335466 (+100848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.317783 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.255306 11588 raft_consensus.cc:515] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:34.343014 8783 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.350134 8037 raft_consensus.cc:2468] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
W20260616 19:59:34.338712 7964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2598) took 3162 ms (client timeout 1696 ms). Trace:
I20260616 19:59:34.333775 8085 leader_election.cc:304] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:34.314117 11544 raft_consensus.cc:2749] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260616 19:59:34.351951 7964 rpcz_store.cc:269] 0616 19:59:31.176510 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:31.198624 (+ 22114us) service_pool.cc:224] Handling call
0616 19:59:34.338317 (+3139693us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000f9454. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:34.338519 (+ 202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2373294336}
I20260616 19:59:34.356330 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:34.361393 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 48347.2us
W20260616 19:59:34.363308 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.344985 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:34.361917 8112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 2002) took 1172 ms. Trace:
I20260616 19:59:34.364343 8041 raft_consensus.cc:3060] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:34.365446 8112 rpcz_store.cc:276] 0616 19:59:33.189092 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:33.189259 (+ 167us) service_pool.cc:224] Handling call
0616 19:59:34.340107 (+1150848us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54002cdb54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:34.361691 (+ 21584us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740505472}
I20260616 19:59:34.366901 8036 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:34.367241 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c266418888004ab0a420a0dcbc5e8477" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
W20260616 19:59:34.368002 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 129709us
I20260616 19:59:34.372131 7969 raft_consensus.cc:2368] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for earlier term 8. Current term is 9.
I20260616 19:59:34.370822 7970 raft_consensus.cc:2468] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:34.314455 11258 raft_consensus.cc:515] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.373898 8040 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:34.370931 11473 raft_consensus.cc:2749] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:34.376806 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2636) took 1019 ms. Trace:
I20260616 19:59:34.378736 7968 rpcz_store.cc:276] 0616 19:59:33.357033 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.339860 (+982827us) service_pool.cc:224] Handling call
0616 19:59:34.376752 (+ 36892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.357120 11451 leader_election.cc:290] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.379971 11157 leader_election.cc:290] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.378284 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:34.377372 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 43668us
I20260616 19:59:34.377588 8042 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 3 FOLLOWER]: Advancing to term 5
W20260616 19:59:34.371788 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 60288.2us
I20260616 19:59:34.385018 8009 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.385821 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:34.376070 7963 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
W20260616 19:59:34.379917 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 61257.2us
I20260616 19:59:34.388875 11469 raft_consensus.cc:2804] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:34.392277 11469 raft_consensus.cc:697] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.393483 11469 consensus_queue.cc:237] T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.395046 8044 raft_consensus.cc:2410] T c266418888004ab0a420a0dcbc5e8477 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260616 19:59:34.396329 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.398104 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.398841 7966 raft_consensus.cc:2393] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 10: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
W20260616 19:59:34.399307 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.399992 7966 raft_consensus.cc:1240] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:34.400339 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.400686 8036 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:34.400995 7968 raft_consensus.cc:3060] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:34.402114 7967 raft_consensus.cc:1240] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:34.404304 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1920) took 1769 ms. Trace:
I20260616 19:59:34.404695 7966 rpcz_store.cc:276] 0616 19:59:32.634557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.399383 (+1764826us) service_pool.cc:224] Handling call
0616 19:59:34.399785 (+ 402us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.400406 (+ 621us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.404229 (+ 3823us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.406143 7966 raft_consensus.cc:1240] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:34.406725 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1933) took 1766 ms. Trace:
I20260616 19:59:34.378715 11344 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:34.407061 11347 raft_consensus.cc:515] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.408589 7935 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.409503 8084 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.384143 11505 raft_consensus.cc:515] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.412462 8045 raft_consensus.cc:2393] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 10: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:34.412686 8085 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:34.413237 8113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42856 (request call id 50) took 1327 ms. Trace:
I20260616 19:59:34.413758 8036 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
W20260616 19:59:34.414932 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.388725 11397 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260616 19:59:34.417761 7964 raft_consensus.cc:2468] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:34.417910 8812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:34.419442 7964 raft_consensus.cc:1240] T 91d1874edf8b4ac187032e20c2e8d509 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:34.419481 8811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.409621 7962 raft_consensus.cc:1240] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:34.402732 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1924) took 1767 ms. Trace:
I20260616 19:59:34.421039 7967 rpcz_store.cc:276] 0616 19:59:32.635460 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.401552 (+1766092us) service_pool.cc:224] Handling call
0616 19:59:34.401940 (+ 388us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.402484 (+ 544us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.402663 (+ 179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.420735 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2622) took 1768 ms. Trace:
I20260616 19:59:34.407058 7966 rpcz_store.cc:276] 0616 19:59:32.640774 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.405500 (+1764726us) service_pool.cc:224] Handling call
0616 19:59:34.405913 (+ 413us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.406485 (+ 572us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.406664 (+ 179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260616 19:59:34.423182 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.422109 7964 rpcz_store.cc:276] 0616 19:59:32.651976 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.418694 (+1766718us) service_pool.cc:224] Handling call
0616 19:59:34.419136 (+ 442us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.420001 (+ 865us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.420683 (+ 682us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.424674 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:34.425583 8817 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 12: Already voted for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in this term.
I20260616 19:59:34.425403 7934 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.428066 8041 raft_consensus.cc:2468] T d38debbe377040e48b9e248db8babc55 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:34.428926 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1935) took 1781 ms. Trace:
I20260616 19:59:34.431084 7963 raft_consensus.cc:1240] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260616 19:59:34.430110 11157 raft_consensus.cc:493] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:34.413623 8113 rpcz_store.cc:276] 0616 19:59:33.085234 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:33.085390 (+ 156us) service_pool.cc:224] Handling call
0616 19:59:34.412854 (+1327464us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54002cdb54. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2dfc33 00007fe8742add0a 00007fe86d831934 00007fe86d8318c2 00007fe86d8317f1 00007fe86d83176c 00007fe86d830a22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:34.413007 (+ 153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014214912}
I20260616 19:59:34.429378 7962 rpcz_store.cc:276] 0616 19:59:32.641224 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.408876 (+1767652us) service_pool.cc:224] Handling call
0616 19:59:34.409237 (+ 361us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.422539 (+ 13302us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.422725 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.426240 11344 raft_consensus.cc:697] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:34.437199 11473 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:34.437320 11544 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260616 19:59:34.437781 11473 raft_consensus.cc:493] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.437894 11544 raft_consensus.cc:493] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.438306 11473 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:34.438447 11544 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Advancing to term 13
I20260616 19:59:34.439142 8084 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:34.442670 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.442781 11347 leader_election.cc:290] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.445853 8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.447715 8817 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:34.448899 7967 raft_consensus.cc:1240] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260616 19:59:34.429131 11505 leader_election.cc:290] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.438373 11312 raft_consensus.cc:2749] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260616 19:59:34.449323 11384 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader election won for term 10
I20260616 19:59:34.455552 11384 raft_consensus.cc:697] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:34.456427 11577 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.437958 11344 consensus_queue.cc:237] T 3dc4c19625e84ce29cb4ca6993524bf1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.456625 11384 consensus_queue.cc:237] T 001495805f9843f0aac7937a5905dc82 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.460354 8085 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:34.461434 8085 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:34.461881 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.462643 11546 raft_consensus.cc:2804] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:34.463811 11380 raft_consensus.cc:2804] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:34.461661 11473 raft_consensus.cc:515] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.462643 8085 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
I20260616 19:59:34.466343 7967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2623) took 1800 ms. Trace:
I20260616 19:59:34.466706 7967 rpcz_store.cc:276] 0616 19:59:32.652323 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.426058 (+1773735us) service_pool.cc:224] Handling call
0616 19:59:34.445914 (+ 19856us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:34.449289 (+ 3375us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.452857 (+ 3568us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.446491 11397 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260616 19:59:34.468858 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 46488.2us
I20260616 19:59:34.470232 8085 leader_election.cc:304] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters:
W20260616 19:59:34.484867 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 49754.3us
W20260616 19:59:34.484687 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 45659.9us
I20260616 19:59:34.485517 8082 leader_election.cc:304] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:34.468185 11545 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:34.447837 11157 raft_consensus.cc:515] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:34.488517 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.471288 11397 raft_consensus.cc:493] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.490455 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:34.499387 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.499873 8817 raft_consensus.cc:2468] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 5.
W20260616 19:59:34.501389 8011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.489964 11546 raft_consensus.cc:697] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.502938 7935 leader_election.cc:304] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.503048 7933 leader_election.cc:304] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:34.504599 7966 raft_consensus.cc:1240] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:34.505733 11321 raft_consensus.cc:2804] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:34.506207 11321 raft_consensus.cc:493] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.506604 11321 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:34.508047 11157 leader_election.cc:290] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.508934 11157 consensus_queue.cc:1059] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:34.510025 11157 raft_consensus.cc:2804] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:34.510601 11157 raft_consensus.cc:697] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.475543 11577 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.516557 11539 consensus_queue.cc:1059] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260616 19:59:34.508247 11576 raft_consensus.cc:2804] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:34.518126 11577 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.491714 11397 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:34.529691 8008 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.530345 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2628) took 1462 ms. Trace:
I20260616 19:59:34.532680 7966 rpcz_store.cc:276] 0616 19:59:33.058124 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.429321 (+1371197us) service_pool.cc:224] Handling call
0616 19:59:34.479011 (+ 49690us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.505104 (+ 26093us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.520500 (+ 15396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.534350 7967 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:34.533164 8042 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:34.531075 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2630) took 1472 ms. Trace:
I20260616 19:59:34.511811 11157 consensus_queue.cc:237] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:34.542371 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 51278.4us
I20260616 19:59:34.544945 8008 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.547465 7964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2629) took 1489 ms. Trace:
I20260616 19:59:34.548537 11515 raft_consensus.cc:3055] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:34.547829 7964 rpcz_store.cc:276] 0616 19:59:33.058441 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.424088 (+1365647us) service_pool.cc:224] Handling call
0616 19:59:34.424847 (+ 759us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:34.436757 (+ 11910us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:34.436787 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:34.445402 (+ 8615us) log.cc:844] Serialized 26 byte log entry
0616 19:59:34.525746 (+ 80344us) raft_consensus.cc:1645] Marking committed up to 10
0616 19:59:34.525783 (+ 37us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.525849 (+ 66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:34.547256 (+ 21407us) raft_consensus.cc:1685] finished
0616 19:59:34.547277 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:34.547413 (+ 136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9070,"raft.queue_time_us":13585,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":59,"thread_start_us":35515,"threads_started":1,"wal-append.queue_time_us":73084}
I20260616 19:59:34.549655 11603 raft_consensus.cc:2764] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260616 19:59:34.515842 11565 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.553118 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43900f33cd8249e2a494e6f9dff7b254" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.548904 11515 raft_consensus.cc:740] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
W20260616 19:59:34.554517 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 76363.9us
I20260616 19:59:34.541725 11505 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:34.563838 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 60862.5us
I20260616 19:59:34.565724 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.566460 8814 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:34.568629 11347 consensus_queue.cc:1059] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260616 19:59:34.568917 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.569621 7966 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:34.569451 11598 consensus_queue.cc:1059] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260616 19:59:34.540436 11588 leader_election.cc:290] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Requested vote from peers 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.536916 7962 rpcz_store.cc:276] 0616 19:59:33.058737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.435736 (+1376999us) service_pool.cc:224] Handling call
0616 19:59:34.441029 (+ 5293us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.441244 (+ 215us) raft_consensus.cc:1534] Early marking committed up to index 8
0616 19:59:34.441657 (+ 413us) log.cc:844] Serialized 14 byte log entry
0616 19:59:34.530768 (+ 89111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:34.530819 (+ 51us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:34.530859 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.530919 (+ 60us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:34.531024 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1684,"prepare.queue_time_us":172,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":36426,"thread_start_us":57167,"threads_started":1,"wal-append.queue_time_us":98098}
I20260616 19:59:34.541365 11577 raft_consensus.cc:3055] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:34.575268 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.575518 11473 leader_election.cc:290] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:34.578737 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 77712.5us
I20260616 19:59:34.578989 7968 raft_consensus.cc:2468] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 7.
I20260616 19:59:34.580084 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.580399 8084 leader_election.cc:304] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.580763 7968 raft_consensus.cc:2393] T 6c06fb426fd349ac813aef7f33c47e85 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:34.580667 11546 consensus_queue.cc:237] T 9b74ddd432434f15bc1a4a3141d4c6aa P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.581666 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.581297 11515 consensus_queue.cc:260] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.582669 11515 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:34.552210 11576 raft_consensus.cc:697] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.586932 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.585664 7935 leader_election.cc:304] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
W20260616 19:59:34.587560 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 95746.1us
I20260616 19:59:34.583513 8814 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:34.589498 8042 raft_consensus.cc:2393] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 11: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:34.553224 11258 leader_election.cc:290] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.588740 11380 raft_consensus.cc:697] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [term 6 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.594085 8084 leader_election.cc:304] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.594750 11380 consensus_queue.cc:237] T 9837c086539a4592bb5aada5872652ca P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.594854 11611 raft_consensus.cc:2749] T e9e060ef9a144de99b92bcba80c31176 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:34.595782 11473 raft_consensus.cc:2749] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260616 19:59:34.597906 8084 leader_election.cc:304] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.598994 11473 raft_consensus.cc:2804] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:34.599375 11473 raft_consensus.cc:493] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.599758 11473 raft_consensus.cc:3060] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:34.601341 11616 raft_consensus.cc:2804] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:34.568619 11367 raft_consensus.cc:2804] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:34.576100 11554 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:34.605705 8041 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: []
I20260616 19:59:34.561970 8816 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 13.
I20260616 19:59:34.570777 11587 leader_election.cc:290] T bd16983859704e7fb1ce248d36c2a8bf P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.609483 7933 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:34.584901 11577 raft_consensus.cc:740] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
W20260616 19:59:34.610329 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.608413 8009 leader_election.cc:304] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
W20260616 19:59:34.612884 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 44648.2us
I20260616 19:59:34.572172 11500 consensus_queue.cc:1048] T 43378ff312f94246811895976a8715d6 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260616 19:59:34.617846 7966 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 11.
I20260616 19:59:34.611394 11410 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:34.605224 11515 raft_consensus.cc:493] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:34.607867 11554 raft_consensus.cc:493] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.622772 11515 raft_consensus.cc:515] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.623421 11347 raft_consensus.cc:3055] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Stepping down as leader of term 7
I20260616 19:59:34.624396 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2633) took 1308 ms. Trace:
I20260616 19:59:34.632448 7966 rpcz_store.cc:276] 0616 19:59:33.315891 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.618608 (+1302717us) service_pool.cc:224] Handling call
0616 19:59:34.623791 (+ 5183us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.624010 (+ 219us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:34.624040 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:34.624083 (+ 43us) raft_consensus.cc:1645] Marking committed up to 3
0616 19:59:34.624119 (+ 36us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.624235 (+ 116us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:34.624340 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.630687 7934 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7; no voters: de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.606817 11505 raft_consensus.cc:515] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:34.641844 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 56291us
I20260616 19:59:34.618224 11367 raft_consensus.cc:697] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:34.641795 11588 raft_consensus.cc:2804] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:34.646751 8114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53916 (request call id 2015) took 1345 ms. Trace:
I20260616 19:59:34.647130 8114 rpcz_store.cc:276] 0616 19:59:33.301154 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:33.362354 (+ 61200us) service_pool.cc:224] Handling call
0616 19:59:34.646395 (+1284041us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400152cd4. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e96da 00007fe8742ab217 00007fe86d823834 00007fe86d8237c2 00007fe86d8236f1 00007fe86d82366c 00007fe86d822922 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:34.646596 (+ 201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273887616}
I20260616 19:59:34.648190 11240 raft_consensus.cc:2749] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260616 19:59:34.648736 7935 leader_election.cc:304] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.651145 7961 raft_consensus.cc:1240] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:34.651723 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 366) took 1154 ms. Trace:
I20260616 19:59:34.652743 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.653235 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22f98752ef9436aa8a128756832bff0" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:34.653869 7965 raft_consensus.cc:3060] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:34.657542 11515 leader_election.cc:290] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.651993 7961 rpcz_store.cc:276] 0616 19:59:33.497090 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.650640 (+1153550us) service_pool.cc:224] Handling call
0616 19:59:34.650979 (+ 339us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.651519 (+ 540us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.651677 (+ 158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.659976 11615 raft_consensus.cc:3055] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Stepping down as leader of term 8
I20260616 19:59:34.631793 11347 raft_consensus.cc:740] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:34.660445 11539 raft_consensus.cc:2804] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 13 FOLLOWER]: Leader election won for term 13
I20260616 19:59:34.660537 11615 raft_consensus.cc:740] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:34.661129 7933 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 51737.6us
I20260616 19:59:34.661410 11588 raft_consensus.cc:697] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:34.662031 7933 leader_election.cc:304] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters:
I20260616 19:59:34.662429 11588 consensus_queue.cc:237] T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.663625 7961 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37398 (request call id 367) took 1166 ms. Trace:
I20260616 19:59:34.664937 11565 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 9 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.661654 11615 consensus_queue.cc:260] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.668478 11384 raft_consensus.cc:2804] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election won for term 9
I20260616 19:59:34.669114 11384 raft_consensus.cc:697] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:34.669579 11607 log.cc:927] Time spent T 30950534b1834f6886db15f85f0abf14 P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260616 19:59:34.670349 7969 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260616 19:59:34.670140 11384 consensus_queue.cc:237] T 5f64cb36408849c989ecda8d19ec4269 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.672642 11539 raft_consensus.cc:697] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [term 13 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:34.673779 11539 consensus_queue.cc:237] T 69c80791d0c44cb0b616bf402ae91b46 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.674785 7963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1919) took 2040 ms. Trace:
I20260616 19:59:34.675263 7963 rpcz_store.cc:276] 0616 19:59:32.634327 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.409269 (+1774942us) service_pool.cc:224] Handling call
0616 19:59:34.421773 (+ 12504us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:34.435527 (+ 13754us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.674511 (+238984us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b540035fcd0. stack: 00007fe86c201c83 00007fe86c201b80 00007fe86bb4f574 00007fe8758946c4 00007fe875894659 00007fe86f24e951 00007fe86f2e4c8d 00007fe8742a9b19 00007fe86d81ed34 00007fe86d81ecc2 00007fe86d81ebf1 00007fe86d81eb6c 00007fe86d81de22 00007fe86d61dc3c 00007fe86d61d076 00007fe86d61ca1f
0616 19:59:34.674684 (+ 173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27824512}
I20260616 19:59:34.668599 7962 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:34.676769 7962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49190 (request call id 1941) took 1595 ms. Trace:
I20260616 19:59:34.676397 11576 consensus_queue.cc:237] T d38debbe377040e48b9e248db8babc55 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.677134 7962 rpcz_store.cc:276] 0616 19:59:33.081542 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.611915 (+1530373us) service_pool.cc:224] Handling call
0616 19:59:34.668435 (+ 56520us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.676528 (+ 8093us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.676720 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.677342 7935 leader_election.cc:304] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.633827 11603 raft_consensus.cc:3055] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:34.664023 7961 rpcz_store.cc:276] 0616 19:59:33.497459 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.660107 (+1162648us) service_pool.cc:224] Handling call
0616 19:59:34.662991 (+ 2884us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.663256 (+ 265us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:34.663285 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:34.663353 (+ 68us) raft_consensus.cc:1645] Marking committed up to 5
0616 19:59:34.663382 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.663438 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:34.663556 (+ 118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.642962 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.689468 7962 raft_consensus.cc:1240] T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260616 19:59:34.680932 7969 raft_consensus.cc:1275] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:59:34.629645 11554 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:34.634516 8011 leader_election.cc:304] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
W20260616 19:59:34.691001 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 72092.6us
W20260616 19:59:34.692198 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 54376.9us
I20260616 19:59:34.693707 8113 raft_consensus.cc:2410] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260616 19:59:34.694854 11587 raft_consensus.cc:2804] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260616 19:59:34.695372 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 60963.2us
I20260616 19:59:34.629895 11410 raft_consensus.cc:515] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.663508 11544 raft_consensus.cc:515] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } attrs { replace: true } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.668066 11615 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:34.697599 7965 raft_consensus.cc:2468] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 5.
I20260616 19:59:34.701546 7965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49196 (request call id 2637) took 1343 ms. Trace:
W20260616 19:59:34.703965 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:34.705616 8785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.704679 7966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2638) took 1325 ms. Trace:
I20260616 19:59:34.707015 7966 rpcz_store.cc:276] 0616 19:59:33.379744 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.636224 (+1256480us) service_pool.cc:224] Handling call
0616 19:59:34.695510 (+ 59286us) raft_consensus.cc:1471] Updating replica for 0 ops
0616 19:59:34.700195 (+ 4685us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:34.700220 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0616 19:59:34.704357 (+ 4137us) raft_consensus.cc:1645] Marking committed up to 8
0616 19:59:34.704412 (+ 55us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.704476 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:34.704615 (+ 139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.688973 11603 raft_consensus.cc:740] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:34.702435 7965 rpcz_store.cc:276] 0616 19:59:33.358441 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.652922 (+1294481us) service_pool.cc:224] Handling call
0616 19:59:34.701475 (+ 48553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260616 19:59:34.711647 8084 leader_election.cc:304] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
W20260616 19:59:34.712945 11563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.011s user 0.009s sys 0.017s
I20260616 19:59:34.712965 8043 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 13: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
W20260616 19:59:34.713511 11563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.122s user 0.010s sys 0.017s
I20260616 19:59:34.713030 8011 leader_election.cc:304] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.703473 11601 raft_consensus.cc:2764] T d38debbe377040e48b9e248db8babc55 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260616 19:59:34.713438 11367 consensus_queue.cc:237] T fd3a274b9cd54eea9bdf95a6e2c9e13a P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.714782 8011 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.714815 11397 raft_consensus.cc:515] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.716847 11240 raft_consensus.cc:2749] T 87bfd6ef4d304dc9899bf6fbd1260a11 P de4c5ce3ea8345ae9068f825b8d60e19 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260616 19:59:34.716398 11515 raft_consensus.cc:2804] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260616 19:59:34.717828 11515 raft_consensus.cc:493] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.718189 11515 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:34.718847 11603 consensus_queue.cc:260] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.720206 11603 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:34.721112 8011 leader_election.cc:304] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:34.722440 8011 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:34.722761 11321 raft_consensus.cc:515] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.723384 11240 raft_consensus.cc:2804] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:34.723795 11240 raft_consensus.cc:493] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.724143 11240 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:34.725852 11505 leader_election.cc:290] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.726258 8114 raft_consensus.cc:1240] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 9. Ops: []
I20260616 19:59:34.726423 11505 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260616 19:59:34.727545 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b96ebb721a6a429ca8c0d058adaf5c6f" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.728440 8111 raft_consensus.cc:2393] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 7: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:34.728726 8113 raft_consensus.cc:1240] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:34.729058 8114 raft_consensus.cc:1240] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 11. Ops: []
I20260616 19:59:34.729604 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74ddd432434f15bc1a4a3141d4c6aa" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:34.730528 11552 log.cc:927] Time spent T f811e1e045a942fa8b77b5c1e2802efd P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 1.718s user 0.003s sys 0.008s
I20260616 19:59:34.731115 11602 raft_consensus.cc:2764] T e22f98752ef9436aa8a128756832bff0 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260616 19:59:34.731065 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38debbe377040e48b9e248db8babc55" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.732583 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.733296 8116 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 7: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:34.734602 8110 raft_consensus.cc:1275] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:59:34.735550 11544 leader_election.cc:290] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 13 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:34.736218 8117 raft_consensus.cc:1275] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260616 19:59:34.737169 11397 leader_election.cc:290] T 02d9effe013f408fad87267b74d4e1ac P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
W20260616 19:59:34.712188 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 119060us
I20260616 19:59:34.738163 8009 leader_election.cc:304] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.738794 8786 leader_election.cc:304] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.740336 11544 raft_consensus.cc:2749] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260616 19:59:34.740437 11498 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:34.745383 11515 raft_consensus.cc:515] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.747200 7935 leader_election.cc:304] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.747515 11515 leader_election.cc:290] T bd096493918740ac8b196f47037f5dc5 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.728284 11621 raft_consensus.cc:2749] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:34.704437 11616 raft_consensus.cc:697] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:34.747974 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.740095 8009 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.749243 11321 leader_election.cc:290] T 001495805f9843f0aac7937a5905dc82 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.748050 11498 raft_consensus.cc:2749] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:34.743637 11240 raft_consensus.cc:515] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
W20260616 19:59:34.751575 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 46688.7us
W20260616 19:59:34.758647 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.749928 11473 raft_consensus.cc:515] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.748958 11344 raft_consensus.cc:2804] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election won for term 7
W20260616 19:59:34.790542 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 50533.3us
I20260616 19:59:34.751746 11565 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:34.702889 11258 consensus_queue.cc:1059] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260616 19:59:34.792675 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.749429 11587 raft_consensus.cc:493] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6827184e3cef42c2ada15b2d31afee4b)
I20260616 19:59:34.751559 11410 leader_election.cc:290] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
W20260616 19:59:34.796599 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.797524 8009 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.794261 11515 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:34.749858 11618 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260616 19:59:34.710279 11347 consensus_queue.cc:260] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.795207 11344 raft_consensus.cc:697] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 LEADER]: Becoming Leader. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
W20260616 19:59:34.803608 8010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:34.797912 11587 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:34.710767 11577 consensus_queue.cc:260] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.790208 11601 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260616 19:59:34.805473 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d1874edf8b4ac187032e20c2e8d509" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.809175 11473 leader_election.cc:290] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:34.809276 11532 consensus_queue.cc:1048] T 00bf27a79bc04305bb057b972a08c316 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260616 19:59:34.807248 11602 consensus_queue.cc:1059] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260616 19:59:34.811564 7968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49196 (request call id 2631) took 1752 ms. Trace:
I20260616 19:59:34.811496 8084 leader_election.cc:304] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.812397 11240 leader_election.cc:290] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.813354 8084 leader_election.cc:304] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.813331 11344 consensus_queue.cc:237] T 69dd551c580f43a4a5e348a7a31398e1 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.814811 8084 leader_election.cc:304] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.816375 11473 raft_consensus.cc:2749] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260616 19:59:34.816661 11577 raft_consensus.cc:3060] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:34.817088 11473 raft_consensus.cc:2804] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260616 19:59:34.817459 11473 raft_consensus.cc:493] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:34.817798 11473 raft_consensus.cc:3060] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:34.801574 11515 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.820791 11515 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:34.812069 7968 rpcz_store.cc:276] 0616 19:59:33.059032 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:34.604536 (+1545504us) service_pool.cc:224] Handling call
0616 19:59:34.621258 (+ 16722us) raft_consensus.cc:1471] Updating replica for 1 ops
0616 19:59:34.621717 (+ 459us) raft_consensus.cc:1534] Early marking committed up to index 0
0616 19:59:34.621749 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0616 19:59:34.622295 (+ 546us) log.cc:844] Serialized 26 byte log entry
0616 19:59:34.688597 (+ 66302us) raft_consensus.cc:1645] Marking committed up to 2
0616 19:59:34.688637 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0616 19:59:34.695918 (+ 7281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0616 19:59:34.811337 (+115419us) raft_consensus.cc:1685] finished
0616 19:59:34.811370 (+ 33us) raft_consensus.cc:1691] UpdateReplicas() finished
0616 19:59:34.811488 (+ 118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2795,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":116,"thread_start_us":65935,"threads_started":1,"wal-append.queue_time_us":169062}
I20260616 19:59:34.815943 8084 leader_election.cc:304] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.803463 11410 raft_consensus.cc:2804] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:34.824859 11410 raft_consensus.cc:697] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:34.825620 11577 raft_consensus.cc:2749] T 138ad2f3a7f24eb8969d70eceea9ed3f P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260616 19:59:34.825847 11410 consensus_queue.cc:237] T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.826364 8084 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.827730 11563 raft_consensus.cc:2749] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260616 19:59:34.828325 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 105977us
I20260616 19:59:34.832820 8011 leader_election.cc:304] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b
I20260616 19:59:34.826433 11577 raft_consensus.cc:2804] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260616 19:59:34.834007 11577 raft_consensus.cc:493] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.834456 11577 raft_consensus.cc:3060] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Advancing to term 13
I20260616 19:59:34.837525 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.837682 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf8afc79e3c417c814ac2fb57557096" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.838193 8119 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 7.
I20260616 19:59:34.838344 8116 raft_consensus.cc:2468] T 2bf8afc79e3c417c814ac2fb57557096 P 6827184e3cef42c2ada15b2d31afee4b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 2.
I20260616 19:59:34.834714 11602 raft_consensus.cc:3055] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Stepping down as leader of term 6
I20260616 19:59:34.835279 11627 raft_consensus.cc:2764] T 9837c086539a4592bb5aada5872652ca P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260616 19:59:34.847769 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 43873.5us
I20260616 19:59:34.852658 11373 raft_consensus.cc:3055] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Stepping down as leader of term 10
I20260616 19:59:34.854452 11373 raft_consensus.cc:740] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:34.855407 11373 consensus_queue.cc:260] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.856464 11373 raft_consensus.cc:3060] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:34.860463 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.861364 8110 raft_consensus.cc:2410] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260616 19:59:34.876534 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd3a274b9cd54eea9bdf95a6e2c9e13a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:34.876564 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 47380.5us
I20260616 19:59:34.879297 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.877722 8084 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:34.884371 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.884891 11158 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:34.888603 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.889840 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.892107 8118 raft_consensus.cc:2393] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 7: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:34.893224 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.844511 11602 raft_consensus.cc:740] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:34.890878 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.898389 8115 raft_consensus.cc:3060] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Advancing to term 9
I20260616 19:59:34.902779 8117 raft_consensus.cc:2468] T 91d1874edf8b4ac187032e20c2e8d509 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 10.
I20260616 19:59:34.902812 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.904096 11240 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:34.904563 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.904665 11240 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.907276 11240 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.907812 11381 raft_consensus.cc:3060] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:34.908774 11472 raft_consensus.cc:3060] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:34.890702 11515 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:34.880337 11616 consensus_queue.cc:237] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:34.895453 11627 raft_consensus.cc:493] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:34.919041 8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:34.921404 8037 raft_consensus.cc:1240] T f7c5954a3dd147cea894b8e392ab2bd6 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 8. Current term is 9. Ops: []
W20260616 19:59:34.922217 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 44620.4us
I20260616 19:59:34.924324 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:34.926030 8043 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 13: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:34.926997 8084 leader_election.cc:304] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters: b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:34.927243 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.928486 8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:34.929283 8816 raft_consensus.cc:1275] T 3e92f5df5ec84f069decf29684f08b06 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260616 19:59:34.930056 11515 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:34.927999 8082 leader_election.cc:304] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:34.935467 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:34.937901 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:35.000980 11623 log.cc:927] Time spent T 02d9effe013f408fad87267b74d4e1ac P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.078s user 0.001s sys 0.001s
I20260616 19:59:35.004545 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.005431 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.024255 7970 raft_consensus.cc:2393] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 12: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:35.005872 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:34.934682 11384 consensus_queue.cc:1048] T 3e92f5df5ec84f069decf29684f08b06 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260616 19:59:35.031054 11563 raft_consensus.cc:493] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.031754 11563 raft_consensus.cc:515] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:35.036764 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 109843us
W20260616 19:59:35.044775 11629 log.cc:927] Time spent T 303be6b16595410aae7147c6026435ff P 6827184e3cef42c2ada15b2d31afee4b: Append to log took a long time: real 0.106s user 0.000s sys 0.000s
I20260616 19:59:35.045033 11546 raft_consensus.cc:2804] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260616 19:59:35.047885 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 119958us
I20260616 19:59:34.922216 11576 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260616 19:59:34.979043 7935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 101503us
W20260616 19:59:35.046921 8784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:35.013191 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.011457 8115 raft_consensus.cc:2468] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 9.
I20260616 19:59:34.929256 8815 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 13: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:34.938534 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.061894 7935 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.063138 7935 leader_election.cc:304] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:35.064839 8816 raft_consensus.cc:1240] T cf8a24d3c225486fa9270926ffe6a40e P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 10 FOLLOWER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:35.065631 8084 leader_election.cc:304] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:35.065972 11620 consensus_queue.cc:1059] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260616 19:59:35.066817 8084 leader_election.cc:304] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:35.067183 11620 raft_consensus.cc:2764] T b96ebb721a6a429ca8c0d058adaf5c6f P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260616 19:59:35.067543 11500 raft_consensus.cc:2749] T c266418888004ab0a420a0dcbc5e8477 P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260616 19:59:35.067864 11620 raft_consensus.cc:2749] T 43900f33cd8249e2a494e6f9dff7b254 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:35.068233 11500 raft_consensus.cc:3060] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:35.068325 11601 raft_consensus.cc:3055] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:35.076884 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:35.070129 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.069957 8111 raft_consensus.cc:2468] T bd16983859704e7fb1ce248d36c2a8bf P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 11.
I20260616 19:59:35.082383 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:35.083086 8115 raft_consensus.cc:2368] T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:59:35.048084 11546 raft_consensus.cc:493] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Starting leader election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
W20260616 19:59:35.089401 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.087774 8110 raft_consensus.cc:2393] T cdd9ba5e3e5c4292b330704225571321 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:35.091531 8115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.099002 8115 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:35.086938 11240 raft_consensus.cc:493] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:35.103483 11240 raft_consensus.cc:515] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.106179 11240 leader_election.cc:290] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:35.069254 11577 raft_consensus.cc:515] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.109934 11539 raft_consensus.cc:493] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.110563 11539 raft_consensus.cc:515] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.114131 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.115013 8037 raft_consensus.cc:2393] T 69c80791d0c44cb0b616bf402ae91b46 P de4c5ce3ea8345ae9068f825b8d60e19 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 13: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:35.109993 11554 raft_consensus.cc:515] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:35.116394 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.117039 8112 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 10.
I20260616 19:59:35.118193 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:35.120055 8082 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.132:37343, user_credentials={real_user=slave}} blocked reactor thread for 70887.6us
I20260616 19:59:35.119642 8042 raft_consensus.cc:1240] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
W20260616 19:59:35.122547 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 55811us
I20260616 19:59:35.123447 8084 leader_election.cc:304] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:34.934544 11627 raft_consensus.cc:515] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:35.091925 11546 raft_consensus.cc:3060] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:35.067225 11603 consensus_queue.cc:1059] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260616 19:59:35.064008 11587 raft_consensus.cc:515] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.129112 8039 raft_consensus.cc:1240] T 5ffc56f0158c4f8a99ad2d5a819bde34 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:35.129211 11587 leader_election.cc:290] T de0955b8f32b489d9b813e4d4ea2cf39 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 5 election: Requested vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:35.076748 11601 raft_consensus.cc:740] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:35.130313 11601 consensus_queue.cc:260] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.130843 8036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8a24d3c225486fa9270926ffe6a40e" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.087085 11545 raft_consensus.cc:2749] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260616 19:59:35.132547 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd16983859704e7fb1ce248d36c2a8bf" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.133287 8045 raft_consensus.cc:2393] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 11: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:35.076764 11515 consensus_queue.cc:1059] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260616 19:59:35.134114 11240 raft_consensus.cc:3055] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Stepping down as leader of term 9
I20260616 19:59:35.134544 11240 raft_consensus.cc:740] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:35.135375 11240 consensus_queue.cc:260] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
W20260616 19:59:35.137921 8011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 290704us
I20260616 19:59:34.936446 11602 consensus_queue.cc:260] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.145345 11240 raft_consensus.cc:3060] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:35.145797 8817 raft_consensus.cc:1240] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:35.147475 11321 raft_consensus.cc:3055] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:35.147894 11321 raft_consensus.cc:740] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:35.148844 11321 consensus_queue.cc:260] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.149940 11321 raft_consensus.cc:3060] T de0955b8f32b489d9b813e4d4ea2cf39 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:35.152866 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.153519 8037 raft_consensus.cc:2468] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:35.119352 11554 leader_election.cc:290] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:35.163137 8008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.165390 7934 leader_election.cc:304] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:35.120175 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8"
I20260616 19:59:35.171813 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.168310 11565 raft_consensus.cc:2749] T f52f3e8cd74b471e9c8b0fc90839f7b0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260616 19:59:35.183189 7935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.179888 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:35.184218 8084 leader_election.cc:304] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.168344 8814 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:35.173220 11373 raft_consensus.cc:2804] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Leader election won for term 7
I20260616 19:59:35.199301 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.200042 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:35.199402 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38333e3e28e64678b975f1c136bcd790" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:35.201464 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 56366.8us
I20260616 19:59:35.179446 11615 raft_consensus.cc:3060] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:35.210680 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.237205 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7747b7f8144d689a26ea1780989278" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.211917 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43378ff312f94246811895976a8715d6" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.243314 8110 raft_consensus.cc:3060] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Advancing to term 13
I20260616 19:59:35.246310 8114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:35.246941 8114 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Advancing to term 6
W20260616 19:59:35.249688 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.199360 8811 raft_consensus.cc:1275] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260616 19:59:35.251500 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.200749 11381 raft_consensus.cc:493] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260616 19:59:35.243572 11639 log.cc:927] Time spent T 00bf27a79bc04305bb057b972a08c316 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.072s user 0.000s sys 0.001s
I20260616 19:59:35.262480 7965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52f3e8cd74b471e9c8b0fc90839f7b0" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.262611 7970 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:59:35.267874 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30950534b1834f6886db15f85f0abf14" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.268642 7964 raft_consensus.cc:2393] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 9: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:35.191314 11380 consensus_queue.cc:1059] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260616 19:59:35.253690 11331 raft_consensus.cc:2804] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 FOLLOWER]: Leader election won for term 5
I20260616 19:59:35.289726 7967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd9ba5e3e5c4292b330704225571321" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.290486 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c25ff952f14593ad9cdf3cf561a445" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.253589 11373 raft_consensus.cc:697] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:35.299674 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a82eaaad5b1444b9ca242cd0aaeed47" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
I20260616 19:59:35.302155 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:35.306396 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:35.310657 11638 log.cc:927] Time spent T 138ad2f3a7f24eb8969d70eceea9ed3f P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
I20260616 19:59:35.267061 11602 raft_consensus.cc:3060] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Advancing to term 7
I20260616 19:59:35.318375 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecd3baa2a052460bb18e02978cb8b80d" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:35.322211 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.323052 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.321020 7961 raft_consensus.cc:2468] T ecd3baa2a052460bb18e02978cb8b80d P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 9.
I20260616 19:59:35.330981 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.263967 8111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ab514ead9d34ff598a454bd28675d12"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:35.333280 8111 raft_consensus.cc:606] T 5ab514ead9d34ff598a454bd28675d12 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:35.331652 8113 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:35.335215 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.335616 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd096493918740ac8b196f47037f5dc5" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.336491 7968 raft_consensus.cc:3060] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:35.308801 11331 raft_consensus.cc:697] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [term 5 LEADER]: Becoming Leader. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:35.271173 11473 raft_consensus.cc:515] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.343981 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.271961 11381 raft_consensus.cc:515] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:35.340756 7966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.446929 7966 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:35.338248 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06bfb4102944cae932c8710569e99d3" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.349910 11627 leader_election.cc:290] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:35.351418 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69c80791d0c44cb0b616bf402ae91b46" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.352177 11554 raft_consensus.cc:493] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:35.351984 11515 consensus_queue.cc:1059] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260616 19:59:35.352027 7962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:35.353279 11645 log.cc:927] Time spent T 3e92f5df5ec84f069decf29684f08b06 P de4c5ce3ea8345ae9068f825b8d60e19: Append to log took a long time: real 0.057s user 0.000s sys 0.001s
W20260616 19:59:35.353744 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:35.452302 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260616 19:59:35.453310 8084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.353891 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
W20260616 19:59:35.454685 7936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.454595 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.455242 7964 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:35.354568 11500 raft_consensus.cc:2749] T 5ffc56f0158c4f8a99ad2d5a819bde34 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260616 19:59:35.357556 8009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.358286 11347 raft_consensus.cc:3060] T cdd9ba5e3e5c4292b330704225571321 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:35.359061 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e92f5df5ec84f069decf29684f08b06" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.360327 8785 leader_election.cc:304] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: b022cac8cb5e46b7a39a748e81e257a7, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:35.361786 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.460582 8119 raft_consensus.cc:1240] T 541911a4cee341d8911dc307cd4a7b44 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 5. Current term is 6. Ops: []
I20260616 19:59:35.460511 8037 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in term 9.
I20260616 19:59:35.362557 7968 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:35.365005 11601 raft_consensus.cc:3060] T f52f3e8cd74b471e9c8b0fc90839f7b0 P b022cac8cb5e46b7a39a748e81e257a7 [term 4 FOLLOWER]: Advancing to term 5
I20260616 19:59:35.366569 8814 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:35.367523 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba7d833edb374b10abaeb128d2084328" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.367763 11539 leader_election.cc:290] T 1f0ed6745c784554bc3af153bcda098a P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:35.369082 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64cb36408849c989ecda8d19ec4269" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
W20260616 19:59:35.473460 8785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 113240us
I20260616 19:59:35.371150 7969 raft_consensus.cc:1275] T 3a82eaaad5b1444b9ca242cd0aaeed47 P b022cac8cb5e46b7a39a748e81e257a7 [term 12 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260616 19:59:35.373430 7963 raft_consensus.cc:1275] T 30950534b1834f6886db15f85f0abf14 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260616 19:59:35.381682 11640 log.cc:927] Time spent T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7: Append to log took a long time: real 0.079s user 0.001s sys 0.000s
I20260616 19:59:35.436192 8112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.478967 7968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.479772 8112 raft_consensus.cc:2410] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 11.
W20260616 19:59:35.436931 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.321718 11344 consensus_queue.cc:1059] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260616 19:59:35.442858 8116 raft_consensus.cc:2393] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 in current term 8: Already voted for candidate 6827184e3cef42c2ada15b2d31afee4b in this term.
I20260616 19:59:35.445256 8110 raft_consensus.cc:2468] T 69c80791d0c44cb0b616bf402ae91b46 P 6827184e3cef42c2ada15b2d31afee4b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 13.
I20260616 19:59:35.446144 8114 raft_consensus.cc:2468] T bd096493918740ac8b196f47037f5dc5 P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 6.
I20260616 19:59:35.448315 7970 raft_consensus.cc:2468] T d06bfb4102944cae932c8710569e99d3 P b022cac8cb5e46b7a39a748e81e257a7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 10.
I20260616 19:59:35.459116 7964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.465395 8039 raft_consensus.cc:1275] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260616 19:59:35.369292 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.477079 8118 raft_consensus.cc:2393] T 5f64cb36408849c989ecda8d19ec4269 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c06f6f0858fe4ce6bd5b55efeb5ba2d8 in current term 9: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:35.477123 11620 raft_consensus.cc:493] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.468413 11554 raft_consensus.cc:515] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.494539 11620 raft_consensus.cc:515] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.495755 8117 raft_consensus.cc:1275] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260616 19:59:35.496651 11620 leader_election.cc:290] T 9f03211770a74604a786608de5062cc3 P b022cac8cb5e46b7a39a748e81e257a7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:35.497145 8117 raft_consensus.cc:1240] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Rejecting Update request from peer b022cac8cb5e46b7a39a748e81e257a7 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260616 19:59:35.497692 8118 raft_consensus.cc:1240] T fd3a274b9cd54eea9bdf95a6e2c9e13a P 6827184e3cef42c2ada15b2d31afee4b [term 7 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:35.498996 8118 raft_consensus.cc:1275] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: Refusing update from remote peer c06f6f0858fe4ce6bd5b55efeb5ba2d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260616 19:59:35.499150 8117 raft_consensus.cc:1240] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Rejecting Update request from peer c06f6f0858fe4ce6bd5b55efeb5ba2d8 for earlier term 9. Current term is 10. Ops: []
I20260616 19:59:35.499302 8118 pending_rounds.cc:85] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Aborting all ops after (but not including) 11
I20260616 19:59:35.499575 8118 pending_rounds.cc:107] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260616 19:59:35.499828 8118 raft_consensus.cc:2889] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260616 19:59:35.500859 7865 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55728 (request call id 200) took 9347 ms. Trace:
I20260616 19:59:35.501825 11554 leader_election.cc:290] T 303be6b16595410aae7147c6026435ff P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887)
I20260616 19:59:35.508518 11554 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.509066 11554 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:35.510613 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:35.511811 11554 leader_election.cc:290] T b14a29bdfa9541db9d3f1e93a225a342 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
I20260616 19:59:35.509353 8784 leader_election.cc:304] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:35.512262 11331 consensus_queue.cc:237] T e22f98752ef9436aa8a128756832bff0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.515713 11563 leader_election.cc:290] T 00bf27a79bc04305bb057b972a08c316 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132:37343)
W20260616 19:59:35.516975 8082 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.517050 11344 raft_consensus.cc:3055] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Stepping down as leader of term 5
I20260616 19:59:35.518069 8814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" is_pre_election: true
I20260616 19:59:35.519548 11615 raft_consensus.cc:2749] T b14a29bdfa9541db9d3f1e93a225a342 P b022cac8cb5e46b7a39a748e81e257a7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260616 19:59:35.521570 11321 raft_consensus.cc:3055] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Stepping down as leader of term 4
I20260616 19:59:35.523490 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.524539 11546 raft_consensus.cc:515] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.477524 11498 raft_consensus.cc:2749] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260616 19:59:35.531453 7966 raft_consensus.cc:2468] T 9918060aeb7f44c892f2ade3f2ac6841 P b022cac8cb5e46b7a39a748e81e257a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:35.557485 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c17f5e21da4e9893d22581a43ceafb" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:35.536746 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 72387.1us
W20260616 19:59:35.542940 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.545852 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.481196 11577 leader_election.cc:290] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 13 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:35.554455 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.556404 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.568756 7969 raft_consensus.cc:3060] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Advancing to term 8
I20260616 19:59:35.568811 8037 raft_consensus.cc:2468] T 7323d4e3a76f4800925df28a0f090072 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 9.
I20260616 19:59:35.556809 8040 raft_consensus.cc:2410] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260616 19:59:35.463647 11576 consensus_queue.cc:1048] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260616 19:59:35.501242 7865 rpcz_store.cc:276] 0616 19:59:26.153718 (+ 0us) service_pool.cc:167] Inserting onto call queue
0616 19:59:26.153941 (+ 223us) service_pool.cc:224] Handling call
0616 19:59:35.500667 (+9346726us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0616 19:59:33.398013 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0616 19:59:33.493473 (+ 95460us) write_op.cc:432] Acquiring schema lock in shared mode
0616 19:59:33.497853 (+ 4380us) write_op.cc:435] Acquired schema lock
0616 19:59:33.503473 (+ 5620us) tablet.cc:662] Decoding operations
0616 19:59:33.519074 (+ 15601us) write_op.cc:620] Acquiring the partition lock for write op
0616 19:59:33.519160 (+ 86us) write_op.cc:641] Partition lock acquired for write op
0616 19:59:33.519188 (+ 28us) tablet.cc:685] Acquiring locks for 13 operations
0616 19:59:33.558063 (+ 38875us) tablet.cc:701] Row locks acquired
0616 19:59:33.561019 (+ 2956us) write_op.cc:260] PREPARE: finished
0616 19:59:33.581011 (+ 19992us) write_op.cc:270] Start()
0616 19:59:33.581142 (+ 131us) write_op.cc:276] Timestamp: P: 1781639973580958 usec, L: 0
0616 19:59:33.581163 (+ 21us) op_driver.cc:348] REPLICATION: starting
0616 19:59:33.632502 (+ 51339us) log.cc:844] Serialized 5314 byte log entry
0616 19:59:33.937463 (+304961us) op_driver.cc:464] REPLICATION: finished
0616 19:59:34.013009 (+ 75546us) write_op.cc:301] APPLY: starting
0616 19:59:34.013157 (+ 148us) tablet.cc:1367] starting BulkCheckPresence
0616 19:59:34.013558 (+ 401us) tablet.cc:1370] finished BulkCheckPresence
0616 19:59:34.019399 (+ 5841us) tablet.cc:1372] starting ApplyRowOperation cycle
0616 19:59:34.021451 (+ 2052us) tablet.cc:1383] finished ApplyRowOperation cycle
0616 19:59:34.021900 (+ 449us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0616 19:59:34.021961 (+ 61us) write_op.cc:312] APPLY: finished
0616 19:59:34.022485 (+ 524us) log.cc:844] Serialized 95 byte log entry
0616 19:59:34.022713 (+ 228us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:34.023042 (+ 329us) write_op.cc:454] Released schema lock
0616 19:59:34.023115 (+ 73us) write_op.cc:341] FINISH: Updating metrics
0616 19:59:34.023735 (+ 620us) write_op.cc:489] Releasing partition, row and schema locks
0616 19:59:34.023786 (+ 51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":964839,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":75124,"apply.run_cpu_time_us":5281,"apply.run_wall_time_us":10692,"num_ops":13,"prepare.queue_time_us":180293,"prepare.run_cpu_time_us":21260,"prepare.run_wall_time_us":402136,"raft.queue_time_us":86175,"raft.run_cpu_time_us":20867,"raft.run_wall_time_us":41889,"replication_time_us":356078,"thread_start_us":266505,"threads_started":4,"wal-append.queue_time_us":170783,"wal-append.run_cpu_time_us":9627,"wal-append.run_wall_time_us":1228855}]]}
I20260616 19:59:35.566881 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.555996 11321 raft_consensus.cc:740] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6827184e3cef42c2ada15b2d31afee4b, State: Running, Role: LEADER
I20260616 19:59:35.573976 8009 leader_election.cc:400] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
I20260616 19:59:35.575214 8119 raft_consensus.cc:1240] T 9f03211770a74604a786608de5062cc3 P 6827184e3cef42c2ada15b2d31afee4b [term 7 LEADER]: Rejecting Update request from peer de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 6. Current term is 7. Ops: []
I20260616 19:59:35.575503 8009 leader_election.cc:403] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260616 19:59:35.576630 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00bf27a79bc04305bb057b972a08c316" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.578296 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d9effe013f408fad87267b74d4e1ac" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.579066 8038 raft_consensus.cc:3060] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:35.576716 8009 leader_election.cc:304] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.585175 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69dd551c580f43a4a5e348a7a31398e1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
W20260616 19:59:35.586676 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 45722.6us
I20260616 19:59:35.582540 8009 leader_election.cc:304] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.473026 11373 consensus_queue.cc:237] T b96ebb721a6a429ca8c0d058adaf5c6f P 6827184e3cef42c2ada15b2d31afee4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.551935 11344 raft_consensus.cc:740] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b022cac8cb5e46b7a39a748e81e257a7, State: Running, Role: LEADER
I20260616 19:59:35.595142 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.583527 8114 raft_consensus.cc:2955] T 7323d4e3a76f4800925df28a0f090072 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Committing config change with OpId 9.10: config changed from index 6 to 10, c06f6f0858fe4ce6bd5b55efeb5ba2d8 (127.6.159.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } attrs { replace: true } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:35.603513 7970 raft_consensus.cc:1275] T 9837c086539a4592bb5aada5872652ca P b022cac8cb5e46b7a39a748e81e257a7 [term 6 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260616 19:59:35.603704 7964 raft_consensus.cc:1275] T 9b74ddd432434f15bc1a4a3141d4c6aa P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer 6827184e3cef42c2ada15b2d31afee4b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:59:35.604686 8037 raft_consensus.cc:2393] T 69dd551c580f43a4a5e348a7a31398e1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b in current term 7: Already voted for candidate b022cac8cb5e46b7a39a748e81e257a7 in this term.
I20260616 19:59:35.607139 8118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c79c08f4d4f61a0e1ff9f6c5ad2c0"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:35.608345 8118 raft_consensus.cc:606] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P 6827184e3cef42c2ada15b2d31afee4b [term 9 LEADER]: Received request to transfer leadership to TS de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:35.611918 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541911a4cee341d8911dc307cd4a7b44" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.611918 7961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e060ef9a144de99b92bcba80c31176" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.580542 11500 raft_consensus.cc:493] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.614887 8044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "001495805f9843f0aac7937a5905dc82" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.615515 8044 raft_consensus.cc:3060] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:35.615559 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef7d39858d46729e29a621de3bc3dd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.585534 11515 raft_consensus.cc:3060] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Advancing to term 9
I20260616 19:59:35.584928 11565 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.594230 11240 raft_consensus.cc:2804] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:35.619122 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dc4c19625e84ce29cb4ca6993524bf1" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.623240 8037 raft_consensus.cc:2468] T 3dc4c19625e84ce29cb4ca6993524bf1 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 10.
I20260616 19:59:35.616874 8039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da6a08976cb46d698d30af739dd2a3b" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.624364 8039 raft_consensus.cc:3060] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:35.629920 8043 raft_consensus.cc:3060] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Advancing to term 6
I20260616 19:59:35.637948 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffc56f0158c4f8a99ad2d5a819bde34" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:35.643841 8786 leader_election.cc:336] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531): Timed out: RequestConsensusVote RPC to 127.6.159.129:42531 timed out after 1.610s (SENT)
I20260616 19:59:35.664345 8814 raft_consensus.cc:1275] T ba7d833edb374b10abaeb128d2084328 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 5 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260616 19:59:35.620743 11498 raft_consensus.cc:2749] T 3a82eaaad5b1444b9ca242cd0aaeed47 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260616 19:59:35.666560 7961 raft_consensus.cc:1275] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Refusing update from remote peer de4c5ce3ea8345ae9068f825b8d60e19: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260616 19:59:35.668790 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "303be6b16595410aae7147c6026435ff" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:35.670274 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 48963.5us
I20260616 19:59:35.672708 8037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9918060aeb7f44c892f2ade3f2ac6841" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.623072 11240 raft_consensus.cc:493] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:35.630184 11565 raft_consensus.cc:515] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.622758 11500 raft_consensus.cc:515] T b14a29bdfa9541db9d3f1e93a225a342 P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } attrs { replace: true } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:35.694286 7969 raft_consensus.cc:2468] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:35.701488 8039 raft_consensus.cc:2468] T 9da6a08976cb46d698d30af739dd2a3b P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 11.
I20260616 19:59:35.713668 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14a29bdfa9541db9d3f1e93a225a342" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.629897 11367 raft_consensus.cc:2804] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260616 19:59:35.702222 8008 leader_election.cc:304] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.692670 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.742637 8045 raft_consensus.cc:2468] T f811e1e045a942fa8b77b5c1e2802efd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:35.688422 8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c06fb426fd349ac813aef7f33c47e85" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.744935 7970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138ad2f3a7f24eb8969d70eceea9ed3f" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
I20260616 19:59:35.747237 11603 consensus_queue.cc:1048] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260616 19:59:35.676402 11473 leader_election.cc:290] T eb7747b7f8144d689a26ea1780989278 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 8 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:35.760455 8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11" candidate_uuid: "6827184e3cef42c2ada15b2d31afee4b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.741048 11367 raft_consensus.cc:493] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260616 19:59:35.756278 11554 raft_consensus.cc:2804] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader election won for term 8
W20260616 19:59:35.763091 8786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.762970 11554 raft_consensus.cc:697] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 LEADER]: Becoming Leader. State: Replica: de4c5ce3ea8345ae9068f825b8d60e19, State: Running, Role: LEADER
I20260616 19:59:35.764307 11554 consensus_queue.cc:237] T 0d19ee585ead4942a40096f7aef3ba50 P de4c5ce3ea8345ae9068f825b8d60e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:35.757055 11546 leader_election.cc:290] T 5c4d4bfa06e04d2d8de1b13263001b1b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 election: Requested vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:35.769812 8043 raft_consensus.cc:2468] T 5eef7d39858d46729e29a621de3bc3dd P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:35.687987 7866 catalog_manager.cc:5696] T 6c06fb426fd349ac813aef7f33c47e85 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:59:35.701135 8084 leader_election.cc:304] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b; no voters: c06f6f0858fe4ce6bd5b55efeb5ba2d8, de4c5ce3ea8345ae9068f825b8d60e19
I20260616 19:59:35.692363 11381 leader_election.cc:290] T 43900f33cd8249e2a494e6f9dff7b254 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129:42531), de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130:43051)
I20260616 19:59:35.736632 11240 raft_consensus.cc:3060] T d06bfb4102944cae932c8710569e99d3 P de4c5ce3ea8345ae9068f825b8d60e19 [term 10 FOLLOWER]: Advancing to term 11
I20260616 19:59:35.774503 7969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e660e68b6c54e9095967ae4b226b0d2" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
I20260616 19:59:35.692282 8816 raft_consensus.cc:1275] T 69c80791d0c44cb0b616bf402ae91b46 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 13 FOLLOWER]: Refusing update from remote peer b022cac8cb5e46b7a39a748e81e257a7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260616 19:59:35.787813 7963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d19ee585ead4942a40096f7aef3ba50" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b022cac8cb5e46b7a39a748e81e257a7" is_pre_election: true
W20260616 19:59:35.786582 8084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.130:43051, user_credentials={real_user=slave}} blocked reactor thread for 85516.7us
I20260616 19:59:35.788443 7963 raft_consensus.cc:2376] T 0d19ee585ead4942a40096f7aef3ba50 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8. Re-sending same reply.
I20260616 19:59:35.779110 7866 catalog_manager.cc:5696] T 56c25ff952f14593ad9cdf3cf561a445 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 7 to 8, leader changed from b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129) to 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131). New cstate: current_term: 8 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:35.772230 11367 raft_consensus.cc:3060] T ecd3baa2a052460bb18e02978cb8b80d P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Advancing to term 10
I20260616 19:59:35.789409 8084 leader_election.cc:304] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
W20260616 19:59:35.790325 8009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 207837us
W20260616 19:59:35.791703 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 220010us
W20260616 19:59:35.792434 8085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.786945 8038 raft_consensus.cc:2468] T 02d9effe013f408fad87267b74d4e1ac P de4c5ce3ea8345ae9068f825b8d60e19 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6827184e3cef42c2ada15b2d31afee4b in term 6.
I20260616 19:59:35.794322 8011 leader_election.cc:304] T 38333e3e28e64678b975f1c136bcd790 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:35.793768 8009 leader_election.cc:304] T 43378ff312f94246811895976a8715d6 P de4c5ce3ea8345ae9068f825b8d60e19 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de4c5ce3ea8345ae9068f825b8d60e19; no voters: 6827184e3cef42c2ada15b2d31afee4b, c06f6f0858fe4ce6bd5b55efeb5ba2d8
I20260616 19:59:35.795069 7866 catalog_manager.cc:5696] T f24cd55ee50d4bcfaee34b2adaa9b08a P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } } }
I20260616 19:59:35.790835 8084 leader_election.cc:304] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.804875 7866 catalog_manager.cc:5696] T 91d1874edf8b4ac187032e20c2e8d509 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:35.804978 8084 leader_election.cc:304] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.818246 7970 raft_consensus.cc:2468] T 138ad2f3a7f24eb8969d70eceea9ed3f P b022cac8cb5e46b7a39a748e81e257a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 7.
I20260616 19:59:35.818609 8084 leader_election.cc:304] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6827184e3cef42c2ada15b2d31afee4b, de4c5ce3ea8345ae9068f825b8d60e19; no voters:
I20260616 19:59:35.648507 11321 consensus_queue.cc:260] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } }
I20260616 19:59:35.823617 8111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f811e1e045a942fa8b77b5c1e2802efd" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.823635 8119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4d4bfa06e04d2d8de1b13263001b1b" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
W20260616 19:59:35.825313 7933 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.789897 11515 raft_consensus.cc:2749] T 9918060aeb7f44c892f2ade3f2ac6841 P de4c5ce3ea8345ae9068f825b8d60e19 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131:40887) with higher term. Message: Invalid argument: T 9918060aeb7f44c892f2ade3f2ac6841 P 6827184e3cef42c2ada15b2d31afee4b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate de4c5ce3ea8345ae9068f825b8d60e19 for earlier term 8. Current term is 9.
W20260616 19:59:35.829841 8083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260616 19:59:35.830050 8045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.860532 8045 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P de4c5ce3ea8345ae9068f825b8d60e19 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b022cac8cb5e46b7a39a748e81e257a7 in term 8.
I20260616 19:59:35.792196 11473 raft_consensus.cc:2749] T 33c17f5e21da4e9893d22581a43ceafb P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260616 19:59:35.829104 7866 catalog_manager.cc:5696] T e8890d947e59455187876dccb01c8c06 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 6 to 8, leader changed from 6827184e3cef42c2ada15b2d31afee4b (127.6.159.131) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 8 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:35.871310 7866 catalog_manager.cc:5696] T 9f03211770a74604a786608de5062cc3 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260616 19:59:35.846688 8118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87bfd6ef4d304dc9899bf6fbd1260a11"
dest_uuid: "6827184e3cef42c2ada15b2d31afee4b"
mode: GRACEFUL
new_leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7"
from {username='slave'} at 127.0.0.1:42856
I20260616 19:59:35.823698 11563 raft_consensus.cc:2804] T f811e1e045a942fa8b77b5c1e2802efd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260616 19:59:35.794561 11258 raft_consensus.cc:2804] T 3dc4c19625e84ce29cb4ca6993524bf1 P 6827184e3cef42c2ada15b2d31afee4b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260616 19:59:35.824666 11381 raft_consensus.cc:493] T 854c79c08f4d4f61a0e1ff9f6c5ad2c0 P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.852982 8038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0955b8f32b489d9b813e4d4ea2cf39" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19"
I20260616 19:59:35.791587 11616 consensus_queue.cc:1048] T c266418888004ab0a420a0dcbc5e8477 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260616 19:59:35.822971 11577 raft_consensus.cc:2804] T 9da6a08976cb46d698d30af739dd2a3b P 6827184e3cef42c2ada15b2d31afee4b [term 11 FOLLOWER]: Leader election won for term 11
I20260616 19:59:35.854804 8110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f0ed6745c784554bc3af153bcda098a" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.860320 11576 raft_consensus.cc:2804] T 5eef7d39858d46729e29a621de3bc3dd P 6827184e3cef42c2ada15b2d31afee4b [term 6 FOLLOWER]: Leader election won for term 6
I20260616 19:59:35.891192 8038 raft_consensus.cc:2393] T de0955b8f32b489d9b813e4d4ea2cf39 P de4c5ce3ea8345ae9068f825b8d60e19 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b022cac8cb5e46b7a39a748e81e257a7 in current term 5: Already voted for candidate de4c5ce3ea8345ae9068f825b8d60e19 in this term.
I20260616 19:59:35.856935 8113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7323d4e3a76f4800925df28a0f090072" candidate_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6827184e3cef42c2ada15b2d31afee4b" is_pre_election: true
I20260616 19:59:35.688692 11344 consensus_queue.cc:260] T 5eef7d39858d46729e29a621de3bc3dd P b022cac8cb5e46b7a39a748e81e257a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } }
I20260616 19:59:35.859110 7964 raft_consensus.cc:1240] T ba7d833edb374b10abaeb128d2084328 P b022cac8cb5e46b7a39a748e81e257a7 [term 5 LEADER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 4. Current term is 5. Ops: []
I20260616 19:59:35.841951 8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f03211770a74604a786608de5062cc3" candidate_uuid: "b022cac8cb5e46b7a39a748e81e257a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" is_pre_election: true
I20260616 19:59:35.845501 8042 raft_consensus.cc:1240] T bd16983859704e7fb1ce248d36c2a8bf P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Rejecting Update request from peer 6827184e3cef42c2ada15b2d31afee4b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260616 19:59:35.884688 7866 catalog_manager.cc:5696] T cf8a24d3c225486fa9270926ffe6a40e P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06f6f0858fe4ce6bd5b55efeb5ba2d8" member_type: VOTER last_known_addr { host: "127.6.159.132" port: 37343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260616 19:59:35.867141 8044 raft_consensus.cc:2410] T 001495805f9843f0aac7937a5905dc82 P de4c5ce3ea8345ae9068f825b8d60e19 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6827184e3cef42c2ada15b2d31afee4b for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260616 19:59:35.870078 7934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 76370.5us
I20260616 19:59:35.873243 8118 raft_consensus.cc:606] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 13 FOLLOWER]: Received request to transfer leadership to TS b022cac8cb5e46b7a39a748e81e257a7
I20260616 19:59:35.785363 11620 consensus_queue.cc:1048] T eb7747b7f8144d689a26ea1780989278 P b022cac8cb5e46b7a39a748e81e257a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260616 19:59:35.877709 8085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.159.129:42531, user_credentials={real_user=slave}} blocked reactor thread for 54322us
I20260616 19:59:35.825558 11498 raft_consensus.cc:493] T f24cd55ee50d4bcfaee34b2adaa9b08a P c06f6f0858fe4ce6bd5b55efeb5ba2d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b022cac8cb5e46b7a39a748e81e257a7)
I20260616 19:59:35.823820 11603 raft_consensus.cc:493] T 6ea1f7e6b1ac4325b6bc7a648cbba327 P b022cac8cb5e46b7a39a748e81e257a7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260616 19:59:35.881312 8110 raft_consensus.cc:2468] T 1f0ed6745c784554bc3af153bcda098a P 6827184e3cef42c2ada15b2d31afee4b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c5ce3ea8345ae9068f825b8d60e19 in term 8.
I20260616 19:59:35.856426 11321 raft_consensus.cc:3060] T ba7d833edb374b10abaeb128d2084328 P 6827184e3cef42c2ada15b2d31afee4b [term 4 FOLLOWER]: Advancing to term 5
W20260616 19:59:35.883908 8784 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.159.131:40887, user_credentials={real_user=slave}} blocked reactor thread for 374611us
I20260616 19:59:35.894666 8118 raft_consensus.cc:614] T 87bfd6ef4d304dc9899bf6fbd1260a11 P 6827184e3cef42c2ada15b2d31afee4b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260616 19:59:35.895118 7866 catalog_manager.cc:5696] T b12fbb557a1a4c488d787ecb4e750fad P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6827184e3cef42c2ada15b2d31afee4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130" port: 43051 } } peers { permanent_uuid: "6827184e3cef42c2ada15b2d31afee4b" member_type: VOTER last_known_addr { host: "127.6.159.131" port: 40887 } } }
I20260616 19:59:35.906826 7866 catalog_manager.cc:5696] T e9e060ef9a144de99b92bcba80c31176 P de4c5ce3ea8345ae9068f825b8d60e19 reported cstate change: term changed from 6 to 9, leader changed from de4c5ce3ea8345ae9068f825b8d60e19 (127.6.159.130) to b022cac8cb5e46b7a39a748e81e257a7 (127.6.159.129). New cstate: current_term: 9 leader_uuid: "b022cac8cb5e46b7a39a748e81e257a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b022cac8cb5e46b7a39a748e81e257a7" member_type: VOTER last_known_addr { host: "127.6.159.129" port: 42531 } } peers { permanent_uuid: "de4c5ce3ea8345ae9068f825b8d60e19" member_type: VOTER last_known_addr { host: "127.6.159.130"